Description
Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Senior Data Engineer. Embark on a journey where your deep-rooted expertise in computer science fundamentals, alongside an intricate understanding of data structures, algorithms, and system design, becomes the cornerstone of innovative solutions. This pivotal role not only demands your proficiency in developing and elevating compute and I/O-intensive applications but also ensures their peak performance and unwavering reliability.
Responsibilities:
- Develop and implement real-time data ingestion and processing systems.
- Design, build, and operationalize large-scale enterprise data solutions and applications.
- Create and manage production data pipelines, from ingestion to consumption, within a big data architecture using PySpark.
- Leverage expertise in Python, Airflow, SQL, and cloud platforms to build robust solutions (strong knowledge of these is essential).
- Translate complex business challenges into scalable and efficient technical solutions.
- Work collaboratively with a high-performing data engineering team, owning the full lifecycle of solution implementation.
Requirements:
- Bachelor's Degree in Computer Science, Information Technology, or a similar discipline.
- 3 - 8 years of professional experience in data engineering or related fields.
- Proven experience with ETL processes, data integration, and handling large-scale datasets using PySpark.
- Strong understanding of data engineering concepts like ETL, near/real-time streaming, data structures, and workflow management.
- Experience with version control tools like Git/GitHub.
- Proficiency in SQL and Data Warehouse concepts.
- Experience working with SQL or NoSQL databases like Cassandra, MongoDB, or HBase.
- Knowledge of AWS technologies such as EMR, RedShift, Kinesis, Lambda, Glue, S3 IAM, CloudWatch, and big data tools like Hadoop/EMR, Hive, and Sqoop.
Similar jobs
Staff Software Engineer (Data Platform)Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Staff Data Engineer. Embark on a journey where your deep-rooted expertise in distributed syst…
About us: Working at Tech Holding isn't just a job, it's an opportunity to be a part of something bigger. We are a full-service consulting firm that was founded on the premise of delivering predictable outcomes and high-…
Job Title: AWS data Engineer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Compan…
Est. 90,000 GBP
Senior Data Engineer Who we are Artefact is a new generation of data service provider, specialising in data consulting and data-driven digital marketing, dedicated to transforming data into business impact across the ent…
Opportunity Overview: We are looking for a Data Engineer to build and optimize scalable data pipelines and analytical datasets on our AWS-based lakehouse platform. You will work with Apache Iceberg, AWS Glue, Athena, dbt…
Job Title: Senior Data EngineerKey Skills: SQL, Databricks, PySpark, Data Engineering, PostgreSQL, AWSExperience: 3+ YOE.Location: Costa Rica, Peru, Colombia, and Bolivia.Mode: Remote. We at Coforge are hiring Senior Dat…
About Us: YipitData is the leading market research and analytics firm for the disruptive economy and most recently raised $475M from The Carlyle Group at a valuation of over $1B. Every day, our proprietary technology ana…
Who We Are Konrad is a next generation digital consultancy. We are dedicated to solving complex business problems for our global clients with creative and forward-thinking solutions. Our employees enjoy a culture built o…
This position will be part of a growing team working towards building world class large scale Big Data architectures. This individual should have a sound understanding of programming principles, experience in programming…
Opportunity Overview: We are seeking a Lead Data Engineer to drive the design and delivery of our cloud-native healthcare data platform. You will lead the implementation of scalable data solutions built on AWS, Apache Ic…
About DevRev At DevRev, we're building the future of work with Computer – your AI teammate. Unlike traditional tools, Computer unifies all your data sources, tools, and workflows into a single AI-ready platform, giving e…
Who we are Artefact is a new generation of data service provider, specialising in data consulting and data-driven digital marketing, dedicated to transforming data into business impact across the entire value chain of or…
Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Staff/Lead Backend Engineer. Embark on a journey where your deep-rooted expertise in computer science fundamentals, alongside an int…
About the Role We are looking for a Senior Data Engineer with strong experience in modern data platforms and cloud-based architectures. The ideal candidate has deep hands-on experience with Snowflake or Databricks, has b…
Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and…
Sigmoid empowers enterprises to make smarter, data-driven decisions by blending advanced data engineering with AI consulting. We collaborate with some of the world’s leading data-rich organizations across sectors such as…
We're looking for a dynamic Senior Data Scientist to join our team, working alongside our founders, financial analysts, product managers, and engineers. This is your chance to get hands-on with massive datasets, crafting…
The Data Engineering team is responsible for building ETL pipelines that populate the Internal Data Platform, which drives analytics that help the company run more efficiently. Our team builds highly performant and scala…
Opportunity Overview: We are seeking a Sr. Data Engineer to help drive the technical vision of our Data Engineering team as we scale our infrastructure, services, and architecture. This role is ideal for someone who blen…
Equal Experts is an innovative consultancy specialising in the delivery of custom software solutions for blue-chip enterprise and public sector clients across a range of industry sectors. We work on important projects fo…
Est. 120,000 PLN
Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable product…
Sigmoid empowers enterprises to make smarter, data-driven decisions by blending advanced data engineering with AI consulting. We collaborate with some of the world’s leading data-rich organizations across sectors such as…
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
Who We Are Konrad is a next generation digital consultancy. We are dedicated to solving complex business problems for our global clients with creative and forward-thinking solutions. Our employees enjoy a culture built o…
Netradyne harnesses the power of Computer Vision and Edge Computing to revolutionize the modern-day transportation ecosystem. We are a leader in fleet safety solutions. With growth exceeding 4x year over year, our soluti…
Join phData, a remote-first data and AI consultancy company with employees across the United States, Latin America, and India. We partner with industry leaders, including Snowflake, AWS, Anthropic, Glean, and dbt, to sol…
KnowBe4 empowers the modern workforce to make smarter security decisions every day. Trusted by more than 70,000 organizations worldwide, KnowBe4 is the pioneer of digital workforce security, securing both AI agents and h…
Est. 133,525 USD
84.51° Overview: 84.51° is a retail data science, insights and media company. We help The Kroger Co., consumer packaged goods companies, agencies, publishers and affiliates create more personalized and valuable experienc…
Job Title: Data Analyst (Python +Pyspark) About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 10…
Who We Are Konrad is a next generation digital consultancy. We are dedicated to solving complex business problems for our global clients with creative and forward-thinking solutions. Our employees enjoy a culture built o…