AWS Data Engineer

EXL Talent Acquisition Team; Bengaluru; On-site; INR 1,200,000 - 2,100,000; Full time

Job summary

EXL Talent Acquisition Team is seeking a Data Engineer in Bengaluru to design and optimize scalable ETL/ELT pipelines, leveraging AWS services for ingestion, transformation, and analytics. You will collaborate with BI, AI/ML, and Infrastructure teams to deliver reliable data platform solutions.

The role requires strong SQL and Python skills, while experience with Airflow and data warehousing concepts will be highly valued.

Qualifications

• 3–5 years of hands-on data engineering experience.
• Design and develop ETL/ELT pipelines.
• SQL and Python proficiency.
• Hands-on AWS experience (Redshift, Lambda, S3, Glue, IAM, CloudWatch).
• Experience with Apache Airflow and data warehousing concepts.
• Knowledge of Git and unit testing practices.

Responsibilities

• Design, develop, and optimize scalable ETL/ELT pipelines for data ingestion and transformation.
• Build and maintain robust data pipelines using AWS cloud services.
• Partner with BI, AI/ML, and Infrastructure teams to deliver data platform solutions.
• Develop SQL queries and Python scripts for data processing, automation, and analytics.
• Create and maintain technical documentation including data flow diagrams and guides.
• Develop unit tests and perform code reviews to ensure quality.
• Monitor, troubleshoot, and optimize pipelines for performance and scalability.
• Manage multiple projects and priorities in a fast-paced environment.
• Follow coding standards, best practices, and data governance guidelines.

Skills

Data Engineering experience

ETL/ELT pipelines

SQL

Python

AWS services

Airflow

Data warehousing concepts

Git

Unit tests

Documentation

Analytical skills

Communication

Job description

Key Responsibilities

• Design, develop, and optimize scalable ETL/ELT pipelines for data ingestion and transformation.
• Build and maintain robust data pipelines using AWS cloud services.
• Partner with Business Intelligence (BI), AI/ML, and Infrastructure teams to deliver reliable and scalable data platform solutions.
• Develop efficient SQL queries and Python scripts for data processing, automation, and analytics.
• Create and maintain technical documentation, including solution designs, data flow diagrams, and operational guides.
• Develop comprehensive unit tests and perform code reviews to ensure high-quality, reliable, and maintainable code.
• Monitor, troubleshoot, and optimize data pipelines to ensure performance, scalability, and reliability.
• Work on multiple projects simultaneously while managing priorities in a fast-paced environment.
• Follow coding standards, best practices, and data governance guidelines.

Required Skills & Qualifications

• 3–5 years of hands‑on experience in Data Engineering.
• Strong experience in designing and developing ETL/ELT pipelines.
• Proficiency in SQL and Python programming.
• Hands‑on experience with AWS services, including:
  • Amazon Redshift
  • AWS Lambda
  • Amazon S3
  • AWS Glue (preferred)
  • AWS IAM
  • Amazon CloudWatch
• Experience with workflow orchestration tools such as Apache Airflow (preferred).
• Understanding of data warehousing concepts and dimensional data modeling.
• Knowledge of version control systems such as Git.
• Experience writing unit tests and following software development best practices.
• Strong analytical, problem‑solving, and debugging skills.
• Excellent communication and documentation skills.
• Ability to manage multiple tasks and work effectively in a collaborative environment.

Preferred Skills

• Experience with CI/CD pipelines and DevOps practices.
• Knowledge of Spark, PySpark, or Databricks is an advantage.
• Exposure to AI/ML data pipelines and feature engineering.
• Familiarity with Agile/Scrum development methodologies.
• AWS Certification (Developer Associate or Data Engineer Associate) is a plus.