AWS Data Engineers @ Capco

Capco; India; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Capco is seeking an experienced Senior AWS Data Engineer to design and implement scalable data pipelines and ETL processes. You will leverage Python, PySpark, Spark SQL, and SQL to build robust data architectures on AWS.

Preferred candidates have Databricks experience, Terraform/ CloudFormation, Git, and Agile delivery experience. You will migrate data from legacy systems to cloud-based solutions and work with AWS data services to optimize data processing workflows.

Qualifications

• Strong experience developing scalable data pipelines and ETL using Python, PySpark, Spark SQL and SQL.
• Experience migrating data to AWS and designing modern cloud-based data architectures.
• Hands-on with AWS data services (S3, Glue, Redshift, Athena, EMR, Lambda, IAM).
• Knowledge of CI/CD, IaC (Terraform or CloudFormation), Git and Agile delivery for automated deployments.
• Databricks knowledge is a plus (certification considered an advantage).
• Ability to provide technical leadership for the team.

Responsibilities

• Develop scalable data pipelines and ETL processes using Python, PySpark, Spark SQL, and SQL.
• Leverage Databricks for data engineering and optimization.
• Support automated deployments with CI/CD, IaC, Git, and Agile practices.
• Migrate data from legacy platforms to AWS and design modern cloud-based data architectures.
• Work with AWS data services (S3, Glue, Redshift, Athena, EMR, Lambda, IAM).

Skills

Python

PySpark

Spark SQL

SQL

AWS Data Engineering

Problem solving

Verbal communication

Interpersonal skills

Technical leadership

Tools

Databricks

Terraform

CloudFormation

Git

CI/CD

Job description

Role & responsibilities

• Strong expertise in Python, PySpark, Spark SQL, and SQL for developing scalable data pipelines and ETL processes.
• Knowledge of Databricks for data engineering and optimization, with Databricks certification considered an advantage.
• Knowledge of CI/CD, Infrastructure as Code (Terraform or CloudFormation), Git, and Agile delivery to support automated and reliable deployments.
• Proven experience migrating data from legacy platforms to AWS, including designing modern cloud based data architectures.
• Hands on experience with AWS data services including S3, Glue, Redshift, Athena, EMR, Lambda, and IAM.

Preferred candidate profile

• Bachelor's degree in computer science, Engineering, or related field, or equivalent work experience.
• Proven 5 -13 years of experience as a AWS Data Engineer or similar role.Strong problem solving and analytical skills.
• Excellent verbal communication skills.
• Good interpersonal skills.
• Ability to provide technical leadership for the team.