AWS Data Engineer

EXL; Bengaluru; On-site; INR 900,000 - 1,300,000; Full time

Job summary

EXL is seeking an experienced Data Engineer to design, build, and optimize scalable ETL/ELT pipelines and data platforms in the AWS ecosystem. You will collaborate with BI, AI/ML, and Infrastructure teams to deliver robust data solutions using SQL and Python.

The role requires 3–5 years of hands-on data engineering experience, strong AWS skills, and a solid grounding in data warehousing concepts. Candidates should be adept at code reviews, testing, and documentation in a collaborative,

Qualifications

• 3–5 years of hands-on experience in Data Engineering.
• Strong experience in designing and developing ETL/ELT pipelines.
• Proficiency in SQL and Python.
• Hands-on experience with AWS services (Redshift, S3, Lambda, Glue, IAM, CloudWatch).
• Experience with workflow orchestration tools such as Apache Airflow (preferred).
• Understanding of data warehousing concepts and dimensional data modeling.
• Knowledge of version control systems such as Git.
• Experience writing unit tests and following software development best practices.
• Strong analytical, problem-solving, and debugging skills.
• Excellent communication and documentation skills.
• Ability to manage multiple tasks and work effectively in a collaborative environment.

Responsibilities

• Design, develop, and optimize scalable ETL/ELT pipelines for data ingestion and transformation.
• Build and maintain robust data pipelines using AWS cloud services.
• Partner with BI, AI/ML, and Infrastructure teams to deliver data platform solutions.
• Develop efficient SQL queries and Python scripts for data processing, automation, and analytics.
• Create and maintain technical documentation, including solution designs, data flow diagrams, and operational guides.
• Develop comprehensive unit tests and perform code reviews to ensure quality.
• Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
• Work on multiple projects simultaneously while managing priorities in a fast-paced environment.
• Follow coding standards, best practices, and data governance guidelines.

Skills

Data Engineering

ETL/ELT pipelines

SQL

Python

AWS

Amazon Redshift

AWS Lambda

Amazon S3

AWS Glue

Git

Airflow

Data Warehousing

Unit Testing

Technical Documentation

Agile/Scrum

Spark/PySpark

Databricks

Education

Bachelor's degree in Computer Science/IT/Engineering

Relevant AWS certifications

Tools

AWS services (Redshift, S3, Lambda, Glue, IAM, CloudWatch)

Git

Job description

Key Responsibilities

• Design, develop, and optimize scalable ETL/ELT pipelines for data ingestion and transformation.
• Build and maintain robust data pipelines using AWS cloud services.
• Partner with Business Intelligence (BI), AI/ML, and Infrastructure teams to deliver reliable and scalable data platform solutions.
• Develop efficient SQL queries and Python scripts for data processing, automation, and analytics.
• Create and maintain technical documentation, including solution designs, data flow diagrams, and operational guides.
• Develop comprehensive unit tests and perform code reviews to ensure high-quality, reliable, and maintainable code.
• Monitor, troubleshoot, and optimize data pipelines to ensure performance, scalability, and reliability.
• Work on multiple projects simultaneously while managing priorities in a fast-paced environment.
• Follow coding standards, best practices, and data governance guidelines.

Required Skills & Qualifications

• 3-5 years of hands-on experience in Data Engineering.
• Strong experience in designing and developing ETL/ELT pipelines.
• Proficiency in SQL and Python programming.
• Hands-on experience with AWS services, including:
  • Amazon Redshift
  • AWS Lambda
  • Amazon S3
  • AWS Glue (preferred)
  • AWS IAM
  • Amazon CloudWatch
• Experience with workflow orchestration tools such as Apache Airflow (preferred).
• Understanding of data warehousing concepts and dimensional data modeling.
• Knowledge of version control systems such as Git.
• Experience writing unit tests and following software development best practices.
• Strong analytical, problem-solving, and debugging skills.
• Excellent communication and documentation skills.
• Ability to manage multiple tasks and work effectively in a collaborative environment.

Preferred Skills

• Experience with CI/CD pipelines and DevOps practices.
• Knowledge of Spark, PySpark, or Databricks is an advantage.
• Exposure to AI/ML data pipelines and feature engineering.
• Familiarity with Agile/Scrum development methodologies.
• AWS Certification (Developer Associate or Data Engineer Associate) is a plus.

Key Competencies

• AWS Data Engineering
• ETL/ELT Development
• SQL & Python
• Amazon Redshift
• AWS Lambda
• Amazon S3
• Data Warehousing
• Unit Testing
• Technical Documentation
• Problem Solving
• Team Collaboration
• Multitasking
• Agile Development

Educational Qualification

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• Relevant AWS certifications will be an added advantage.