AWS Senior Data Engineers

Tata Consultancy Services; Pune District, Chennai District, Bengaluru; On-site; INR 2,800,000 - 4,200,000; Full time

Job summary

Tata Consultancy Services in India is seeking an experienced AWS Senior Data Engineer to design and build scalable cloud-based data platforms on AWS. The role focuses on ETL/ELT pipelines, data lakes, and analytics across enterprise systems.

The ideal candidate has hands-on experience with AWS Lambda, MWAA, S3, Glue, Redshift, EMR, Snowflake, and Python, and will optimize big data workloads while ensuring security and cost governance. Pune/Bengaluru-based roles with remote options may apply.

Qualifications

• Hands-on experience designing scalable cloud-based data platforms on AWS.
• Proficiency with PySpark for batch/stream processing and Spark tuning.
• Experience building and optimizing ETL/ELT pipelines on AWS data services.

Responsibilities

• Design and implement distributed data processing on AWS EMR with PySpark.
• Build robust ETL/ELT pipelines from diverse sources (files, RDBMS, APIs, streaming).
• Optimize Spark jobs, tune EMR clusters, and implement data lake patterns on S3.
• Orchestrate workflows with Step Functions or Airflow and automate CI/CD.
• Ensure cost governance, observability, and security controls across data platforms.
• Collaborate with analytics teams to expose curated datasets to Redshift/Athena/Glue.

Skills

AWS Lambda

MWAA

AWS S3

AWS Glue

Amazon Redshift

Amazon EMR

Snowflake

Python

PySpark

SQL

ETL / ELT Development

Data Lake Architecture

AWS Step Functions

CI/CD Pipelines

Job description

Greetings from TCS!!!
TCS presents an excellent opportunity for AWS Senior Data Engineers

Role- AWS Senior Data Engineers

Desired Experience Range- 8-12 Years

Location- Chennai, Bangalore, Kolkata, Pune

Job Description

We are looking for an experienced AWS Senior Data Engineer with strong expertise in designing and building scalable cloud-based data platforms on AWS. The ideal candidate should have hands-on experience with AWS Lambda, Managed Airflow (MWAA), AWS S3, AWS Glue, Amazon Redshift, Amazon EMR, Snowflake, and Python. The role involves developing high-performance ETL/ELT pipelines, implementing data lake solutions, optimizing big data workloads, and enabling analytics across enterprise platforms.

Roles & Responsibilities

• Design and implement distributed data processing on AWS EMR using PySpark for batch/stream workloads.
• Build robust, scalable ETL/ELT pipelines integrating diverse data sources (files, RDBMS, APIs, streaming).
• Optimize Spark jobs (partitioning, caching, broadcast joins), and tune EMR clusters (instance types, autoscaling, bootstrap actions).
• Implement data lake patterns on S3 with appropriate file formats (Parquet/ORC), schema evolution, and table management.
• Orchestrate workflows using AWS Step Functions, Airflow, or EMR managed workflow, and automate CI/CD.
• Ensure cost governance (spot instances, right-sizing, scaling policies) and observability (CloudWatch, Spark metrics).
• Enforce security & compliance (IAM, KMS, encryption at rest/in transit, VPC, SGs, Glue Catalog access controls).
• Collaborate with analytics teams to expose curated datasets to Redshift, Athena, Glue, or downstream BI tools.
• Troubleshoot performance bottlenecks and production issues; implement resiliency patterns and SLAs.
• Document architecture, standards, runbooks; mentor junior engineers.

Mandatory Skills

• AWS Lambda
• Managed Airflow (MWAA)
• AWS S3
• AWS Glue
• Amazon Redshift
• Amazon EMR
• Snowflake
• Python
• PySpark
• SQL
• ETL / ELT Development
• Data Lake Architecture
• AWS Step Functions
• CI/CD Pipelines