AWS Senior Data Engineers - S

Tata Consultancy Services; Pune District, Chennai District, Bengaluru; On-site; INR 2,800,000 - 4,200,000; Full time

Job summary

Tata Consultancy Services is seeking an AWS Senior Data Engineer to design and build scalable cloud-based data platforms on AWS. You will lead ETL/ELT development, data lake patterns, and analytics exposure to downstream tools like Redshift and Athena.

Responsibilities include optimizing Spark workloads, orchestrating workflows with MWAA/Step Functions, and ensuring cost governance and security across data pipelines.

Qualifications

• 8+ years of experience in AWS data engineering and cloud-based data platforms.
• Hands-on design and development of ETL/ELT pipelines on AWS.
• Experience with data lake architectures and multi-source data integration.

Responsibilities

• Design distributed data processing on AWS EMR with PySpark for batch/stream workloads.
• Build scalable ETL/ELT pipelines from diverse data sources (files, RDBMS, APIs).
• Tune Spark/EMR performance, manage clusters, autoscaling and bootstrap actions.
• Implement data lake patterns on S3 with Parquet/ORC, schema evolution, table management.
• Orchestrate workflows with Step Functions, Airflow, or EMR workflow and CI/CD automation.
• Ensure cost governance and observability with CloudWatch and Spark metrics.
• Enforce security and access controls (IAM, KMS, VPC, Glue Catalog).
• Collaborate with analytics teams to expose curated datasets to Redshift, Athena, Glue.
• Troubleshoot performance bottlenecks and ensure production resiliency and SLAs.
• Document architecture, standards, and runbooks; mentor junior engineers.

Skills

Python

PySpark

SQL

ETL/ELT Development

Data Lake Architecture

CI/CD Pipelines

Tools

AWS Lambda

Managed Airflow (MWAA)

AWS S3

AWS Glue

Amazon Redshift

Amazon EMR

Snowflake

Python

SQL

AWS Step Functions

Job description

Greetings from TCS!!!
TCS presents an excellent opportunity for AWS Senior Data Engineers

Role- AWS Senior Data Engineers

Desired Experience Range- 8-12 Years

Location- Chennai, Bangalore, Kolkata, Pune

Job Description -

We are looking for an experienced AWS Senior Data Engineer with strong expertise in designing and building scalable cloud-based data platforms on AWS. The ideal candidate should have hands‑on experience with AWS Lambda, Managed Airflow (MWAA), AWS S3, AWS Glue, Amazon Redshift, Amazon EMR, Snowflake, and Python. The role involves developing high‑performance ETL/ELT pipelines, implementing data lake solutions, optimizing big data workloads, and enabling analytics across enterprise platforms.

Roles & Responsibilities-

• Design and implement distributed data processing on AWS EMR using PySpark for batch/stream workloads.
• Build robust, scalable ETL/ELT pipelines integrating diverse data sources (files, RDBMS, APIs, streaming).
• Optimize Spark jobs (partitioning, caching, broadcast joins), and tune EMR clusters (instance types, autoscaling, bootstrap actions).
• Implement data lake patterns on S3 with appropriate file formats (Parquet/ORC), schema evolution, and table management.
• Orchestrate workflows using AWS Step Functions, Airflow, or EMR managed workflow, and automate CI/CD.
• Ensure cost governance (spot instances, right‑sizing, scaling policies) and observability (CloudWatch, Spark metrics).
• Enforce security & compliance (IAM, KMS, encryption at rest/in transit, VPC, SGs, Glue Catalog access controls).
• Collaborate with analytics teams to expose curated datasets to Redshift, Athena, Glue, or downstream BI tools.
• Troubleshoot performance bottlenecks and production issues; implement resiliency patterns and SLAs.
• Document architecture, standards, runbooks; mentor junior engineers.

Mandatory Skills

• AWS Lambda
• Managed Airflow (MWAA)
• AWS S3
• AWS Glue
• Amazon Redshift
• Amazon EMR
• Snowflake
• Python
• PySpark
• SQL
• ETL / ELT Development
• Data Lake Architecture
• AWS Step Functions
• CI/CD Pipelines