Data engineer- ALM

Capco; Pune District; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Capco, a Wipro company, is a global technology and management consulting firm recruiting a Senior Data DevOps Engineer. You will manage Cloudera CDP/CDH environments, optimize data processing workloads, and build scalable CI/CD pipelines.

The role focuses on deploying PySpark/Python solutions, ensuring security and governance, and maintaining high availability across enterprise-scale platforms within our Pune, India delivery centers.

Qualifications

• Hands-on experience with Cloudera Hadoop/CDP platforms.
• Strong expertise in HDFS, Spark/PySpark, Python, Hive, YARN, Hue, and CDE.
• Hands-on experience with Jenkins, GitHub/Bitbucket, and CI/CD practices.
• Good knowledge of SQL and performance tuning.
• Experience with Checkmarx or similar SAST tools.
• Strong Linux administration and scripting skills.
• Excellent troubleshooting and problem-solving abilities.

Responsibilities

• Administer and support Cloudera CDP/CDH platforms, including HDFS, Hive, Spark, YARN, Hue, and CDE.
• Develop, deploy, and optimize PySpark and Python-based data processing solutions.
• Build and maintain CI/CD pipelines using Jenkins and GitHub/Bitbucket.
• Integrate security and code quality tools such as Checkmarx into delivery pipelines.
• Write and optimize SQL queries for Hive and Impala environments.
• Monitor platform health, perform upgrades, troubleshoot issues, and ensure high availability.
• Implement security controls, access management, and governance best practices.

Skills

DevOps mindset

Problem-solving

Scripting

Tools

Cloudera Hadoop/CDP

HDFS

Spark/PySpark

Python

Hive

YARN

Hue

CDE

Jenkins

GitHub/Bitbucket

CI/CD pipelines

SQL

Performance tuning

Checkmarx

Linux scripting

Troubleshooting

Job description

“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery.

WHY JOIN CAPCO?

You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.

MAKE AN IMPACT

Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.

#BEYOURSELFATWORK

Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.

CAREER ADVANCEMENT

With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.

DIVERSITY & INCLUSION

We believe that diversity of people and perspective gives us a competitive advantage.

MAKE AN IMPACT

Job Title: Data Engineer

Job Summary:

Seeking a skilled Senior Data DevOps Engineer having experience in Cloudera platforms, data engineering, and DevOps automation. The ideal candidate will manage and optimize Cloudera environments, build CI/CD pipelines, and support enterprise-scale data processing workloads.

Key Responsibilities

Administer and support Cloudera CDP/CDH platforms, including HDFS, Hive, Spark, YARN, Hue, and CDE. Develop, deploy, and optimize PySpark and Python-based data processing solutions. Build and maintain CI/CD pipelines using Jenkins and GitHub/Bitbucket. Integrate security and code quality tools such as Checkmarx into delivery pipelines. Write and optimize SQL queries for Hive and Impala environments. Monitor platform health, perform upgrades, troubleshoot issues, and ensure high availability. Implement security controls, access management, and governance best practices.

Required Skills Hands on experience with Cloudera Hadoop/CDP platforms. Strong expertise in HDFS, Spark/PySpark, Python, Hive, YARN, Hue, and CDE. Hands-on experience with Jenkins, GitHub/Bitbucket, and CI/CD practices. Good knowledge of SQL and performance tuning. Experience with Checkmarx or similar SAST tools. Strong Linux administration and scripting skills. Excellent troubleshooting and problem-solving abilities