Data Engineer with Scala and Spark/ Pyspark
Bengaluru, Karnataka, India · Posted 2 months agoDescription
About Us
“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery.
WHY JOIN CAPCO?
You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.
MAKE AN IMPACT
Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.
#BEYOURSELFATWORK
Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.
CAREER ADVANCEMENT
With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.
DIVERSITY & INCLUSION
We believe that diversity of people and perspective gives us a competitive advantage.
MAKE AN IMPACT
Job Title: Data Engineer
Position: Data Engineer
Experience: 4+ Years
Work Mode: Hybrid (Capco Office)
Key Responsibilities
- Design, develop, and optimize large-scale data processing applications using Scala, Apache Spark, and Java.
- Build and maintain high-performance data pipelines capable of processing 80–90 million records daily with a focus on scalability, reliability, and efficiency.
- Develop and integrate Native APIs and data services to support business-critical applications and analytics platforms.
- Collaborate with cross-functional teams to gather requirements and deliver robust data engineering solutions.
- Optimize Spark jobs, data workflows, and distributed computing processes to ensure high throughput and low latency.
- Implement best practices for data quality, monitoring, governance, and operational excellence.
- Troubleshoot and resolve performance bottlenecks across data processing and ingestion pipelines.
- Participate in code reviews and contribute to engineering standards, architecture decisions, and continuous improvement initiatives.
Required Skills & Experience
- Strong hands-on experience in Scala, Apache Spark, and Java.
- Proven experience building and supporting large-scale distributed data processing systems handling tens of millions of records daily.
- Expertise in developing and consuming Native APIs and microservices.
- Strong understanding of Spark architecture, performance tuning, partitioning, caching, and optimization techniques.
- Experience with data modeling, ETL/ELT processes, and large-scale batch and streaming data pipelines.
- Solid understanding of distributed systems, concurrency, and high-volume data processing.
- Experience with SQL and relational/non-relational databases.
- Strong debugging, analytical, and problem-solving skills.
Preferred Qualifications
- Experience working in enterprise-scale data environments within Financial Services, Payments, or FinTech domains.
- Exposure to cloud platforms (AWS, Azure, or GCP) and containerized deployments.
- Familiarity with Kafka, Airflow, Hadoop ecosystem, or similar big data technologies.
- Experience with CI/CD pipelines, DevOps practices, and Agile methodologies.
What We're Looking For
- Engineers passionate about solving complex data challenges at scale.
- Professionals who can design highly performant systems capable of processing and transforming massive datasets efficiently.
- Team players who thrive in fast-paced, collaborative environments and take ownership of end-to-end delivery.
Similar jobs
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
Job Title: AWS data Engineer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Compan…
Job Title: Data Analyst (Python +Pyspark) About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 10…
Job Title: Data Analyst (Python +Pyspark) About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 10…
Job Title: Data modeler About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies f…
Job Title: Data Scientist with R programming About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top…
Job Title: Data(Basel 3.1) About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companie…
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
Job Title: Data modeler About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies f…
Job Title: Data Modeller About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies…
Est. 29,000 EUR
Joining Capco means joining an organisation that is committed to an inclusive working environment where you’re encouraged to #BeYourselfAtWork. We celebrate individuality and recognize that diversity and inclusion, in al…
Job Title: Business Data Analyst About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Co…
Job Title: Data Analyst (Python +Pyspark) About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 10…
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
Job Title: Senior Data Scientist About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Co…
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
Job Title: Python Engineer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companie…
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
Job Title: Hadoop Admin About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies f…
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
Job Title: Python Developer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Compani…
Est. 120,000 GBP
Principal Azure Data Engineer (Databricks) Location: London (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent Lead the architecture of data platforms driving the future of financial services. The R…
Job Title: Data Engineer - Python and Azure Data Bricks About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been…
Job Title: Java Microservices Developer - Bangalore About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ran…
Please note we are looking to speak to individuals already based in the Doha as a first priority. Capco, a Wipro company, is a global technology and management consultancy specializing in driving digital transformation i…
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
Capco, a Wipro company, is a global technology and management consultancy specializing in driving digital transformation in the financial services industry. With a growing client portfolio comprising of over 100 global o…
Est. 80,000 GBP
Senior Azure Data Engineer (Databricks) Location: London (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent Engineer advanced data solutions at scale in a world-class consulting environment. The Rol…