Lead Data Engineer - AWS Glue

NetConnectGlobal; Delhi; On-site; INR 1,500,000 - 2,000,000; Full time

Job summary

NetConnectGlobal is seeking a highly skilled Lead Data Engineer to drive large-scale data transformation initiatives. The ideal candidate will leverage expertise in AWS Data Services, PySpark, and various NoSQL databases to design and optimize data pipelines.

This role requires a minimum of 5 years of Data Engineering experience, including at least 2 years in a lead position. Strong communication and mentoring skills are essential, along with a focus on data quality and governance.

Qualifications

• 5+ years of Data Engineering experience with 2+ years in a Lead role.
• Strong hands-on expertise in AWS Glue, S3, Redshift, and EMR.
• Advanced knowledge of PySpark/Spark for large-scale data processing.

Responsibilities

• Design, develop, and optimize large-scale data pipelines using AWS services and PySpark.
• Build and maintain scalable ETL/ELT frameworks and data ingestion pipelines.
• Ensure data quality, governance, metadata management, and security compliance.

Skills

AWS Glue

Amazon S3

Amazon Redshift

PySpark / Apache Spark

Data Engineering

ETL / ELT Pipelines

Couchbase

DynamoDB

MongoDB

CI/CD Pipelines

Tools

GitLab

AWS CodePipeline

Job description

Job Summary

NetConnect Global is hiring a Lead Data Engineer for a reputed client. We are looking for a highly skilled professional with expertise in AWS Data Services, PySpark, Data Warehousing, NoSQL Databases, and Enterprise Data Platforms to drive large-scale data transformation initiatives.

Key Skills

• AWS Glue
• Amazon S3
• Amazon Redshift
• Amazon EMR
• PySpark / Apache Spark
• Data Engineering
• ETL / ELT Pipelines
• Couchbase
• DynamoDB
• MongoDB
• Data Warehousing
• CI/CD Pipelines
• GitLab
• AWS CodePipeline
• Data Governance
• Data Quality
• Microservices

Responsibilities

• Design, develop, and optimize large-scale data pipelines using AWS services and PySpark.
• Build and maintain scalable ETL/ELT frameworks and data ingestion pipelines.
• Design and implement ODS layers using NoSQL databases such as Couchbase, DynamoDB, or MongoDB.
• Develop high-performance data processing solutions using Spark.
• Implement CI/CD pipelines and deployment automation.
• Ensure data quality, governance, metadata management, and security compliance.
• Support microservices‑based architectures and enterprise data platforms.
• Collaborate with business, analytics, and engineering teams to deliver data-driven solutions.
• Mentor junior engineers and drive engineering best practices.

Required Experience

• 5+ years of Data Engineering experience with 2+ years in a Lead role.
• Strong hands‑on expertise in AWS Glue, S3, Redshift, and EMR.
• Advanced knowledge of PySpark/Spark for large-scale data processing.
• Experience with Couchbase, DynamoDB, MongoDB, or similar NoSQL databases.
• Strong understanding of ETL/ELT design patterns and data architecture.
• Experience with GitLab, AWS CodePipeline, or similar CI/CD tools.
• Exposure to Retail, POS, MMS, or Transaction Data environments is highly preferred.
• Excellent communication and stakeholder management skills.