AWS Data Engineer - BNG/CHN/HYD-Permanent Role with MNC

MNC Group; Hyderabad, Chennai District, Bengaluru; Hybrid; INR 2,600,000 - 4,200,000; Full time

Job summary

MNC Group is seeking a Senior Data Engineer to lead the design and development of scalable data pipelines and lakehouse architectures on AWS. You will enable real-time and batch analytics across structured and semi-structured data sources, while ensuring governance and security.

The role involves building data lakes, enabling federated querying with Athena/Redshift Spectrum, and integrating SQL and NoSQL sources. Collaboration with analytics teams and adherence to Agile processes are essential.

Qualifications

• Strong experience with AWS Glue, PySpark, Athena and Glue Data Catalog.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using AWS Glue, Glue Studio, and Glue Workflows.
• Build data lakes and lakehouses using S3, Lake Formation, and Glue Data Catalog.
• Enable interactive querying using Athena, Redshift Spectrum, and Presto.
• Ingest and process streaming data using Amazon Kinesis (Data Streams, Firehose).
• Integrate data from SQL and NoSQL sources (RDS, DynamoDB, MongoDB).
• Implement data governance, lineage, and access control using Lake Formation, IAM, and resource policies.
• Optimize performance, cost, scalability, and storage efficiency across pipelines and catalog layers.
• Implement data quality checks, validation frameworks, and observability mechanisms.
• Collaborate with data scientists and analytics teams for curated datasets for ML, BI, reporting.
• Ensure compliance with security, encryption, auditing, and regulatory standards.
• Develop architectural diagrams, data models, and transformation logic; maintain operational docs.
• Participate in Agile/Scrum processes; troubleshoot production issues and drive improvements.

Skills

PySpark

ETL/ELT pipelines

AWS Glue

Athena

SQL

NoSQL design

Tools

AWS Glue

Glue Studio

Glue Workflows

S3

Lake Formation

Glue Data Catalog

Athena

Redshift Spectrum

Kinesis

RDS

DynamoDB

MongoDB

Presto

Job description

About the Role:

We are looking for a highly skilled and motivated Senior Data Engineer to lead the design and development of scalable, secure, and high-performance data pipelines and lakehouse architectures on AWS. You will play a critical role in building and optimizing our data infrastructure, enabling real-time and batch analytics across structured and semi-structured data sources.

Key Responsibilities:

• Design, develop, and maintain scalable ETL/ELT pipelines using AWS Glue, Glue Studio, and Glue Workflows
• Build and manage data lakes and lakehouses using S3, Lake Formation, and Glue Data Catalog
• Enable interactive and federated querying using Athena, Redshift Spectrum, and Presto
• Ingest and process streaming data using Amazon Kinesis (Data Streams, Firehose)
• Integrate and transform data from SQL and NoSQL sources (e.g., RDS, DynamoDB, MongoDB)
• Implement data governance, lineage, and access control using Lake Formation, IAM, and resource policies
• Optimize performance, cost, scalability, and storage efficiency across pipelines, catalog layers, and compute frameworks
• Implement data quality checks, validation frameworks, and observability mechanisms to ensure reliable, trustworthy datasets
• Work with data scientists and analytics teams to deliver curated, clean, and production-ready datasets for ML, BI, and reporting
• Ensure compliance with security, encryption, auditing, and regulatory standards
• Develop architectural diagrams, data models, transformation logic, and operational documentation
• Collaborate in Agile/Scrum processes involving sprint planning, estimation, and retrospectives
• Troubleshoot production issues, perform root cause analysis, and drive continuous improvement of data systems

Required Skills & Qualifications:

Category

Requirements

ETL & Data Pipelines

Strong experience with AWS Glue, PySpark, Athena, and Glue Data Catalog

Data Storage

Proficient with Amazon S3, Redshift Spectrum, RDS, DynamoDB, and Lake Formation

Streaming

Hands-on with Amazon Kinesis (Data Streams, Firehose)

Data Modeling

Strong knowledge of:

• SQL & NoSQL design
• Partitioning strategies
• Schema evolution and versioning
• Performance tuning and query optimization
• Data normalization vs. denormalization strategies