Lead Data Engineer AWS

Merkle; New Delhi; On-site; INR 2,500,000 - 4,200,000; Full time

Job summary

Merkle is seeking an AWS-focused Technical Lead in Data Engineering with 6 to 10 years of experience to design and deliver scalable, secure data platforms on AWS. You will work closely with global teams and clients to implement enterprise-grade data solutions, focusing on data pipelines, quality, and performance.

The role emphasizes technical ownership, hands-on development, and collaboration with architects, product owners, and stakeholders to translate business requirements into robust AWS

Qualifications

• Hands-on experience with SQL and Python for data transformations and analytics.
• Strong knowledge of AWS data services (S3, Glue, Athena, Redshift) and data pipelines.
• Experience building scalable batch and streaming data pipelines.
• Experience with distributed processing (Spark / PySpark).
• Proven data modeling, performance tuning, and cost optimization skills.

Responsibilities

• Data Platform Design & Development on AWS.
• Design and implement AWS-based data engineering solutions for enterprise standards.
• Build and optimize batch and streaming data pipelines.
• Develop SQL-driven transformations and Python-based data pipelines for analytics.
• Design efficient data models for performance, scalability, and cost.

Skills

SQL

Python

Spark/PySpark

Data Engineering

AWS Data Services

Education

Bachelor's or Master's in Computer Science or related field

Tools

AWS

Job description

Job Description

The purpose of this role is to provide technical guidance and suggest improvements in development processes. Develop required software features, achieving timely delivery in compliance with the performance and quality standards of the company.

Business Title

Technical Lead

Years of Experience

Min 6 and max upto 10.

Job Description

Looking for a hands‑on AWS Technical Lead – Data Engineering with 6 to 10 years of total experience to design and deliver scalable, secure, and high‑performance data platforms on AWS.
This role focuses on strong individual contribution with technical ownership, working closely with global teams, architects, and clients to deliver enterprise‑grade data engineering solutions. The position requires deep expertise in AWS data services, SQL, and Python, and the ability to build and optimize reliable data pipelines for analytics and business use cases.

Key responsibiltes

• Data Platform Design & Development
• Design and implement AWS‑based data engineering solutions aligned to enterprise standards
• Build and optimize batch and streaming data pipelines using AWS native and open‑source tools
• Develop SQL‑driven transformations and Python‑based data pipelines for analytics use cases
• Design efficient data models for performance, scalability, and cost effectiveness
• Delivery & Quality Ownership
• Own data engineering deliverables from development through production support
• Perform performance tuning, cost optimization, and capacity planning
• Troubleshoot complex data pipeline and production issues, including root‑cause analysis
• Ensure solutions meet requirements for security, reliability, and scalability
• Collaboration & Client Engagement
• Work closely with architects, product owners, and client stakeholders
• Translate business and analytics requirements into robust AWS data engineering solutions
• Provide clear technical inputs, estimates, and implementation trade‑offs
• Contribute to solution discussions and technical design reviews
• Engineering Best Practices
• Follow and contribute to coding standards, documentation, and data engineering best practices
• Participate in code reviews and continuous improvement initiatives
• Ensure adherence to AWS, security, and compliance guidelines

Must have skills

• Cloud & Data Engineering (AWS)
• Strong hands‑on experience with AWS data services, including:
• Amazon S3, AWS Glue, Athena, Redshift
• Experience designing cloud‑native data lakes and data warehouse architectures on AWS
• Deep understanding of batch and streaming data pipelines
• Experience building scalable, fault‑tolerant data ingestion and transformation workflows
• SQL & Python (Mandatory)
• Strong SQL expertise
• Writing complex SQL for transformations, aggregations, performance tuning, and analytics
• Hands‑on experience handling large‑scale datasets in Redshift / Athena
• Strong Python programming skills (mandatory) for data engineering use cases
• PySpark / Spark‑based processing
• Building reusable ETL components, utilities, and data pipelines
• Strong understanding of data modeling, transformations, and performance optimization
• Data Processing & Engineering
• Proven hands‑on experience with distributed processing frameworks such as Spark / PySpark
• Experience working with structured, semi‑structured, and unstructured data
• Solid understanding of schema design, partitioning, and query optimization
• DevOps & Platform Engineering
• Experience with Infrastructure as Code using Terraform and/or CloudFormation
• Hands‑on experience building and maintaining CI/CD pipelines for data platforms
• Exposure to containerized workloads (Docker, ECS/EKS where applicable to data workloads)
• Collaboration & Ownership
• Strong ownership mindset for solution quality, performance, and production stability
• Excellent communication skills to collaborate with architects, DevOps, QA, and business stakeholders

Good to have skills

• Experience with real‑time/streaming technologies (Kinesis, Kafka, MSK)
• Exposure to Lakehouse architectures and modern data platform patterns
• Experience integrating AWS data platforms with BI and analytics tools
• Knowledge of data governance, data quality, and metadata management
• Familiarity with FinOps practices for optimizing AWS data platform costs
• Exposure to marketing, customer, or analytics data domains (CDP / MarTech)
• Experience working in Agile delivery models with global delivery exposure

Education Qulification

1. Bachelor’s or Master’s degree in Computer Science, Information Systems, Data Engineering, or a related field.

Certification If Any

• AWS Data Analytics / Solutions Architect
• Any two of the above
• Databricks, Snowflake, or other cloud data platform certifications are a plus.

Shift timing

12 PM to 9 PM and / or 2 PM to 11 PM - IST time zone

Location

DGS India - Pune - Indiqube Orchid

Brand

Merkle

Time Type

Full time

Contract Type

Permanent