Description
Company Background
Our client is a global leader in access solutions and smart home technologies. With over a decade of innovation in the IoT space, the company is a pioneer in connected home solutions, including garage door openers, gate systems, and smart access management tools. One of their flagship innovations is the platform, which allows users to control and monitor their home access systems through connected devices. Headquartered in Oak Brook, Illinois, our client serves both residential and commercial customers worldwide.
Project Description
The specialist will join a team building modern cloud-based data platforms using Databricks, Python, and PySpark. The role focuses on designing and implementing scalable data pipelines based on the Medallion Architecture (Bronze, Silver, and Gold), as well as developing low-latency data services that integrate with multiple external systems through APIs.
Technologies
- Python
- PySpark
- Databricks
- SQL
- REST APIs
- Git
- Delta Lake
- Databricks Workflows
- Apache Kafka
- AWS / Azure
What You'll Do
- Design, build, and maintain scalable data pipelines using Databricks;
- Design and implement API integrations to ingest data from multiple internal and external systems;
- Implement and maintain data pipelines based on the Medallion Architecture (Bronze, Silver, and Gold) to support reliable and scalable analytics;
- Build and optimize batch and real-time data processing pipelines to meet performance, latency, and reliability requirements;
- Develop reusable data transformation frameworks using Python and PySpark;
- Optimize Spark jobs and Databricks workloads for performance and cost efficiency;
- Ensure data quality through validation, monitoring, and automated testing;
- Collaborate with architects, product owners, and engineering teams to design scalable data solutions;
- Troubleshoot production issues and continuously improve pipeline reliability and observability;
- Document data flows, architectural decisions, and operational procedures;
Job Requirements
- 5+ years of experience in Data Engineering;
- Strong experience with Databricks;
- Advanced proficiency in Python, PySpark and storing SQL skills;
- Experience designing and implementing data pipelines based on the Medallion Architecture (Bronze, Silver, and Gold);
- Experience building scalable ETL/ELT data pipelines;
- Experience integrating data from REST APIs and third-party services;
- Experience developing real-time or near-real-time data pipelines;
- Experience optimizing Spark workloads for performance and scalability;
- Understanding of data quality, governance, and monitoring best practices;
- Strong problem-solving and communication skills, with English proficiency at B2 level or higher;
- Bachelor's degree in Computer Science or a related field;
Nice to Have
- Experience with Delta Lake and Delta Live Tables (DLT);
- Experience with Databricks Workflows;
- Experience with Unity Catalog;
- Experience with Apache Kafka or other streaming technologies;
- Experience with Azure Data Factory, AWS Glue, or similar data orchestration tools;
- Experience implementing CI/CD processes for data pipelines;
- Experience working in Agile teams;
What Do We Offer
The global benefits package includes:
- Technical and non-technical training for professional and personal growth;
- Internal conferences and meetups to learn from industry experts;
- Support and mentorship from an experienced employee to help you professional grow and development;
- Health insurance;
- Sports activities to promote a healthy lifestyle;
- Flexible work options, including remote and hybrid opportunities;
- Referral program for bringing in new talent;
- Work anniversary program and additional vacation days.
Similar jobs
Project Description We are looking for an experienced Data Engineer with deep expertise in Databricks to join our advanced analytics and data engineering team. The ideal candidate will play a key role in designing, build…
Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable product…
Est. 84,000 PLN
Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable product…
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
P-1348 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best dat…
P-1348 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best dat…
CSQ227R215 As a Staff Designated Engineer and tech subject matter expert, you will partner closely with our Field and Engineering teams to deliver high-touch specialised support and tailored technical solutions for Datab…
CSQ227R215 As a Senior Designated Engineer and tech subject matter expert, you will partner closely with our Field and Engineering teams to deliver high-touch specialized support and tailored technical solutions for Data…
P-1577 As a Sr. Forward Deployed Engineer (FDE) you will work with customers to build and productionize solutions to their data & AI challenges using the Databricks platform. You will own the architecture, lead desig…
Est. 271,425 USD
Databricks builds the world's leading data and AI platform, used by tens of thousands of organizations to turn raw data into decisions that matter. The Lakeflow Engineering team owns the entirety of the ETL product line:…
Est. 90,000 GBP
REQ ID: FEQ427R344 Location: London, UK Recruiter: Kanwal Matharu At Databricks, our core values are at the heart of everything we do; creating a culture of proactiveness and a customer-centric mindset guides us to creat…
Est. 260,200 USD
FEQ227R317 At Databricks, we are on a mission to empower our customers to solve the world's toughest data problems by utilizing the the Databricks Data Intelligence Platform. As a Delivery Solutions Architect (DSA), you…
Est. 120,000 EUR
(REQ ID: CSQ427R209)Databricks is redefining what organisations can achieve with data and AI. As the unified platform powering the world's most ambitious data-driven enterprises, we sit at the intersection of cloud migra…
Est. 213,750 USD
FEQ327R420 As a Data Engineering and Warehousing Specialist Solutions Architect (SSA), you will lead the advanced technical strategy for your customers — owning complex architecture discussions, driving platform adoption…
Est. 90,000 EUR
P-1321 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. W…
Est. 213,000 USD
P-1381 At Databricks, we are passionate about enabling Data & AI teams to solve the world's toughest problems - from making the next mode of transportation a reality to accelerating the development of medical breakth…
Est. 213,750 USD
While candidates in the listed location(s) are encouraged for this role, candidates in other locations will be considered. #LI-REMOTE As a Strategic Solutions Architect on the Digital Natives team, you will shape the fut…
Est. 226,000 USD
P-186 At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best data an…
Est. 213,750 USD
FEQ427R143 As an Infrastructure & Platform Specialist Solutions Architect (SSA), you will serve as a trusted technical advisor, guiding enterprise customers through the architecture, administration, and security of t…
Est. 90,000 GBP
The Role As a Sr. Solutions Engineer, you will independently lead technical engagements for customers, owning discovery, solution design, and platform demonstrations. You are a builder who can code, architect, and presen…
Est. 226,000 USD
P-186 At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best data an…
Est. 185,750 USD
P-987 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We…
Est. 165,000 USD
P-1388 As a Sr. Manager of the Data & AI Support Engineering team, you will lead and manage a team of Technical Solutions Engineers responsible for driving deep technical resolutions for complex customer issues acros…
Est. 213,750 USD
FEQ427R143 As an Infrastructure & Platform Specialist Solutions Architect (SSA), you will serve as a trusted technical advisor, guiding enterprise customers through the architecture, administration, and security of t…
Est. 195,500 USD
P-59 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We…
Est. 214,700 USD
P-988 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We…
Est. 120,000 SEK
CSQ227R121 Locations - Sweden, Denmark, Finland (Hybrid/Remote) At Databricks, we are on a mission to empower our customers to solve the world's toughest data problems with the Databricks Data Intelligence Platform. As a…
Est. 120,000 GBP
REQ ID: FEQ127R254 Location: Hybrid – London As a Senior Forward Deployed Engineering - Architect you will work with customers to build and productionize solutions to their data & AI challenges using the Databricks p…
Est. 213,750 USD
CSQ227R17 At Databricks, we are on a mission to empower our customers to solve the world's toughest data problems by utilizing the the Databricks Data Intelligence Platform. As a Delivery Solutions Architect (DSA), you w…
(P-1384) About the Team At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical…