Senior Data Engineer

dunnhumby; Gurugram District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

A data-driven marketing firm is looking for a talented Senior Data Engineer to design and optimize data processing pipelines for retail media measurement. The ideal candidate will have 7–9 years of experience, strong skills in SQL and PySpark, and familiarity with cloud data solutions like Redshift or BigQuery. This role offers an opportunity to grow expertise in modern data engineering while collaborating with cross-functional teams to enhance data solutions that drive impactful insights.

Qualifications

• 7–9 years of experience as a Data Engineer.
• Experience with scalable architecture and distributed data processing systems.
• Strong programming skills in SQL and PySpark.
• Hands-on experience with cloud-based data stores such as Redshift or BigQuery (preferred).

Responsibilities

• Design, develop, and optimize data processing pipelines for retail media measurement.
• Work with cross-functional teams to build robust data solutions.
• Implement scalable data pipelines and enhance data reliability.

Skills

SQL

PySpark

Apache Spark

Hive

Python

Git

Tools

Docker

Kubernetes

Redshift

BigQuery

Job description

Job Title: Senior Data Engineer

Retail Media is transforming how advertisers connect with consumers through personalized and targeted campaigns across retailers' digital and physical touchpoints. Retail Media Measurement plays a pivotal role in ensuring the effectiveness of these campaigns, driving value for advertisers, retailers, and consumers alike.

This role focuses on designing, building, and scaling solutions that enable the accurate measurement of retail media campaigns across various channels. By providing actionable insights, it empowers stakeholders to optimize media investments, improve ROI, and enhance the overall customer experience.

Job Summary

We are looking for a talented and motivated Senior Data Engineer to contribute to the design, development, and optimization of real‑time and batch data processing pipelines for our retail media measurement solution. In this role, you will work with tools such as Python, Apache Spark, and streaming frameworks to process and analyze data, supporting near‑real‑time decision‑making for critical business applications in the retail media space.

You will collaborate with cross‑functional teams, including Data Scientists, Analysts, and Senior Engineers, to build robust and efficient data solutions. As a Data Engineer, you will focus on implementing scalable data pipelines under the guidance of senior team members while gaining hands‑on experience with streaming and batch processing systems. Your contributions will help ensure the reliability and performance of our data infrastructure, driving impactful insights for the business.

What We Expect From You

Experience

• 7–9 years of experience as a Data Engineer.
• Prior experience working with scalable architecture and distributed data processing systems.

Technical Expertise

• Strong programming skills in SQL and PySpark.
• Proficiency in big data solutions such as Apache Spark and Hive.
• Experience with big data workflow orchestrators like Argo Workflows.
• Hands‑on experience with cloud‑based data stores such as Redshift or BigQuery (preferred).
• Familiarity with cloud platforms, preferably GCP or Azure.

Development Practices

• Strong programming skills in Python, with experience in frameworks like FastAPI or similar API frameworks.
• Proficiency in unit testing and ensuring code quality.Hands‑on experience with version control tools like Git.

Optimization & Problem Solving

• Ability to analyze complex data pipelines, identify performance bottlenecks, and suggest optimization strategies.
• Work collaboratively with infrastructure teams to ensure a robust and scalable platform for data science workflows.
• Excellent problem‑solving skills and the ability to work effectively in a team environment.
• Strong communication skills to collaborate across teams and share technical insights.

Nice To Have

• Experience with microservices architecture, containerization using Docker, and orchestration tools like Kubernetes.
• Exposure to MLOps practices or machine learning workflows using Spark.
• Understanding of logging, monitoring, and alerting for production‑grade big data pipelines.

This role is ideal for someone eager to grow their expertise in modern data engineering practices while contributing to impactful projects in a collaborative environment.