Aws Data Engineer (Blr/Chn/Hyd/Kochi/Kol/Pune )

Tata Consultancy Services; Hyderabad, Pune District, Bengaluru; On-site; INR 1,500,000 - 2,300,000; Full time

Job summary

Tata Consultancy Services is seeking an experienced AWS Data Engineer to design, develop, and maintain scalable data pipelines and cloud-native ETL solutions on AWS. The role demands hands-on expertise with AWS data services, data integration, real-time processing, and modern data engineering practices to build reliable data platforms.

You will design ingestion pipelines, implement data quality measures, and collaborate with stakeholders to deliver data solutions.

Qualifications

• Minimum 6 years of experience in Data Engineering and Data Integration.

Responsibilities

• Design, develop, and optimize data ingestion, transformation, and integration pipelines using AWS services.
• Build scalable ETL/ELT frameworks using Python, PySpark, and AWS Glue.
• Develop data pipelines to process structured and semi-structured data from multiple sources including databases, files, APIs, and streaming platforms.
• Implement data quality checks and validation processes to ensure reliable data delivery.
• Develop and maintain cloud-based data solutions using AWS Glue, Lambda, Step Functions, CloudFormation, S3, EC2, IAM.
• Design and maintain data models supporting analytics and reporting solutions.
• Work with PostgreSQL, Redshift, and relational databases.
• Develop and optimize SQL queries, stored procedures, and transformations.
• Build orchestration pipelines using AWS Step Functions, AWS Glue Workflows, and Apache Airflow.
• Participate in Agile development processes and sprint-based delivery.
• Collaborate with architects, developers, product teams, and business stakeholders.

Skills

Strong analytical and problem-solving

Excellent communication

Ability to work under tight timelines

Tools

AWS Glue

AWS Lambda

AWS CloudFormation

PySpark

Python

SQL

AWS Step Functions

Amazon S3

Amazon EC2

AWS IAM

Apache Airflow

Amazon Redshift

Kafka Streaming

Matillion

SSIS

Visual Studio

PostgreSQL

Job description

We are seeking an experienced AWS Data Engineer with strong expertise in Python, PySpark, SQL, AWS Glue, Lambda, CloudFormation, and Data Engineering. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and cloud-native ETL solutions on AWS.

The role requires hands-on experience with AWS data services, data integration frameworks, real-time processing, and modern data engineering practices to build highly scalable and reliable data platforms.

Key Responsibilities

Data Engineering & ETL Development

• Design, develop, and optimize data ingestion, transformation, and integration pipelines using AWS services.
• Build scalable ETL/ELT frameworks using Python, PySpark, and AWS Glue.
• Develop data pipelines to process structured and semi-structured data from multiple sources including databases, files, APIs, and streaming platforms.
• Implement data quality checks and validation processes to ensure reliable data delivery.

AWS Cloud Development

• Develop and maintain cloud-based data solutions using:
  
  • AWS Glue
  • AWS Lambda
  • AWS Step Functions
  • AWS CloudFormation
  • Amazon S3
  • Amazon EC2
  • AWS IAM
• Build highly available, scalable, and secure cloud-native data platforms.
  
• Automate infrastructure provisioning and deployment using Infrastructure as Code (IaC).
  

Data Modeling & Database Development

• Design and maintain data models supporting analytics and reporting solutions.
• Work with PostgreSQL, Redshift, and relational databases.
• Develop and optimize SQL queries, stored procedures, and transformations.
• Implement best practices for database performance tuning and scalability.

Real-Time & Pipeline Engineering

• Develop workflows for batch and real-time data processing.
  
• Build orchestration pipelines using:
  
  • AWS Step Functions
  • AWS Glue Workflows
  • Apache Airflow
• Support event-driven architectures and streaming data use cases.
  

Agile Delivery & Collaboration

• Participate in Agile development processes and sprint-based delivery.
• Collaborate with architects, developers, product teams, and business stakeholders.
• Conduct peer reviews and ensure adherence to engineering best practices.
• Support client demos, testing activities, and production deployments.

Required Skills

• Minimum 6 years of experience in Data Engineering and Data Integration.
  
• 3+ years of hands-on experience in:
  
  • AWS Glue
  • AWS Lambda
  • AWS CloudFormation
  • PySpark
• Strong Python programming experience.
  
• Strong SQL development and optimization skills.
  
• Experience with:
  
  • AWS Step Functions
  • AWS IAM
  • Amazon S3
  • Amazon EC2
• Experience developing ETL/ELT pipelines and data integration frameworks.
  
• Strong understanding of Data Engineering concepts and best practices.
  
• Experience implementing data models and analytical platforms.
  
• Experience building scalable and high-performance workflows.
  

Preferred Skills

• Apache Airflow.
• Amazon Redshift.
• Kafka Streaming.
• Matillion.
• SSIS.
• Visual Studio.
• Real-time data processing frameworks.
• Enterprise Data Warehouse solutions.

Desired Candidate Profile

• Strong analytical and problem-solving skills.
• Excellent verbal and written communication abilities.
• Experience working on multiple concurrent projects.
• Ability to collaborate with technical and non-technical stakeholders.
• Strong ownership mindset and proactive approach.
• Ability to work effectively under tight timelines.