Lead Data Engineer - ETL

Srijan Technologies PVT LTD; Gurugram District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Srijan Technologies PVT LTD is seeking a highly skilled Data Engineering Lead in Gurugram, India. The role involves leading the design and optimization of scalable data pipelines using AWS services to support analytical use cases.

With a focus on data quality and performance, the candidate will utilize tools like AWS Glue and Apache Spark, while providing guidance to the data engineering team. This position emphasizes collaboration across teams to ensure efficient data integration and management.

Qualifications

• 5+ years of experience in data engineering.
• Strong hands-on experience with AWS data services.
• Proficiency in PySpark / Spark for data processing.

Responsibilities

• Design and implement scalable ETL/ELT pipelines.
• Develop a centralized Operational Data Store using NoSQL technologies.
• Conduct peer reviews and provide technical leadership.

Skills

AWS data services expertise

PySpark / Spark proficiency

ETL/ELT design patterns

Data governance knowledge

Problem-solving skills

Tools

AWS Glue

Amazon S3

Amazon Redshift

Couchbase

Apache Spark

Job description

About Us

We turn customer challenges into growth opportunities. Material is a global strategy partner to the world’s most recognizable brands and innovative companies. Our people around the globe thrive by helping organizations design and deliver rewarding customer experiences.

We use deep human insights, design innovation and data to create experiences powered by modern technology. Our approaches speed engagement and growth for the companies we work with and transform relationships between businesses and the people they serve.

Srijan, a Material company, is a renowned global digital engineering firm with a reputation for solving complex technology problems using their deep technology expertise and leveraging strategic partnerships with top-tier technology partners. Be a part of an Awesome Tribe.

Job Description

We are seeking a highly skilled and experienced Data Engineering Lead with strong expertise in AWS data services and retail data ecosystems. In this role, you will lead the design, development, and optimization of scalable data pipelines responsible for ingesting and transforming data from MMS (Merchandise Management Systems) and POS (Point of Sale) systems into a centralized Operational Data Store (ODS) to support downstream applications and analytics use cases.

You will play a critical role in building a robust, high-performance data platform using AWS-native services, ensuring data quality, reliability, and real-time or near real-time availability for business operations.

Responsibilities

• Design and implement scalable ETL/ELT pipelines to ingest data from MMS / POS or third-party systems into AWS-based data platforms.
• Build and maintain a centralized Operational Data Store (ODS) using NoSQL technologies such as Couchbase or similar, optimized for low-latency access.
• Develop and optimize data processing workflows using Apache Spark / PySpark and AWS services such as AWS Glue and Amazon EMR.
• Create denormalized, API-ready data models aligned with downstream microservices and application consumption patterns.
• Implement idempotent processing, CDC merge (upsert) strategies, and data reconciliation mechanisms to ensure consistency across batch and streaming pipelines.
• Leverage Amazon S3, Amazon Redshift, or Amazon Aurora for storage and querying of structured and semi-structured data.
• Implement data ingestion patterns (batch and streaming) using tools like Amazon Kinesis or AWS Lambda where applicable.
• Apply performance tuning and optimization techniques to improve pipeline efficiency, scalability, and cost-effectiveness.
• Define and enforce data governance, data quality, and metadata management standards across the data platform.
• Collaborate with DevOps teams to design and maintain CI/CD pipelines using tools like AWS CodePipeline, GitLab, or similar.
• Conduct peer reviews and provide technical leadership and mentorship to the data engineering team.
• Collaborate with microservices and application teams to ensure seamless integration with the ODS via APIs and event streams.

Requirements

• 5+ years of experience in data engineering, with at least 2+ years in a lead role.
• Strong hands-on experience with AWS data services such as AWS Glue, Amazon S3, Amazon Redshift, and Amazon EMR.
• Proficiency in PySpark / Spark for large-scale data processing and optimization.
• Experience designing and implementing ODS layers using NoSQL databases, preferably Couchbase, or similar (DynamoDB, MongoDB).
• Strong expertise in ETL/ELT design patterns, data ingestion, and transformation pipelines.
• Experience working with MMS, POS, or retail transaction data is highly preferred.
• Hands-on experience with CI/CD pipelines (GitLab, AWS CodePipeline, or similar).
• Good understanding of data governance, data quality, and metadata frameworks.
• Experience supporting microservices architectures with data platforms.
• Strong problem-solving skills and the ability to optimize complex data workflows.
• Excellent communication and stakeholder management skills.
• Ability to work in a fast-paced, agile environment and manage multiple priorities.