Senior Data Engineer - Customer One

McKinsey & Company; Gurugram District; On-site; INR 3,500,000 - 6,500,000; Full time

Job summary

McKinsey & Company in Gurugram, India seeks an experienced Data Engineer to design, build, and deploy modular ML models for customer value management and personalization. You will work across data pipelines, experimentation, and production, delivering scalable solutions.

The role emphasizes hyperparameter tuning, data wrangling, and collaboration with a global team, with cloud, Databricks, and GenAI experience valued.

Qualifications

• Degree in Computer Science or Software Engineering, or equivalent experience preferred
• 6+ years of experience with practical knowledge of software engineering concepts and best practices in a data engineering context, including unit testing for data transformations (e.g., pandas and PySpark), data validation (e.g. Great Expectations)
• Familiarity with distributed computing frameworks (e.g., Spark, ibis), containerization (e.g., Docker), and analytics libraries (e.g. pandas, numpy, scikit-learn)
• Experience with cloud platforms (e.g. AWS, Azure, GCP) and Databricks a plus
• Experience with Git Hub
• Proficiency in writing clean, maintainable, scalable, and robust code in Python in a professional setting
• Proven experience building data pipelines for advanced analytics use cases
• Experience with Generative AI technologies, including interfacing with foundational models (e.g., OpenAI GPT-4), designing AI Agents, and prompt engineering for various content creation tasks

Responsibilities

• Design, build, and deploy robust, modular, scalable, and reproducible machine learning models tailored to customer value management and personalization applications and adapt and fine-tune models to deliver significant business value
• Develop new machine learning models and algorithms to address emerging business challenges and opportunities in customer value management
• Perform hyperparameter tuning, model validation, and performance optimization to enhance model accuracy and efficiency
• Curate, wrangle, and prepare data for use in advanced analytics models, ensuring high-quality input features

Skills

Python

PySpark

Spark

Pandas

Docker

AWS

Azure

GCP

Great Expectations

Unit testing

Data pipelines

GitHub Actions

Generative AI

OpenAI GPT-4

GitHub

Education

B.Sc. in Computer Science or Software Engineering

Tools

Databricks

Kedro

ibis

GitHub Actions

Job description

Who You’ll Work With

Driving lasting impact and building long-term capabilities with our clients is not easy work. You are the kind of person who thrives in a high performance/high reward culture - doing hard things, picking yourself up when you stumble, and having the resilience to try another way forward.

Who You’ll Work With

Driving lasting impact and building long-term capabilities with our clients is not easy work. You are the kind of person who thrives in a high performance/high reward culture - doing hard things, picking yourself up when you stumble, and having the resilience to try another way forward. In return for your drive, determination, and curiosity, we’ll provide the resources, mentorship, and opportunities you need to become a stronger leader faster than you ever thought possible. Your colleagues—at all levels—will invest deeply in your development, just as much as they invest in delivering exceptional results for clients. Every day, you’ll receive apprenticeship, coaching, and exposure that will accelerate your growth in ways you won’t find anywhere else. When you join us, you will have:

• Continuous learning: Our learning and apprenticeship culture, backed by structured programs, is all about helping you grow while creating an environment where feedback is clear, actionable, and focused on your development. The real magic happens when you take the input from others to heart and embrace the fast-paced learning experience, owning your journey.
• A voice that matters: From day one, we value your ideas and contributions. You’ll make a tangible impact by offering innovative ideas and practical solutions, all while upholding our unwavering commitment to ethics and integrity. We not only encourage diverse perspectives, but they are critical in driving us toward the best possible outcomes.
• Global community: With colleagues across 65+ countries and over 100 different nationalities, our firm’s diversity fuels creativity and helps us come up with the best solutions for our clients. Plus, you’ll have the opportunity to learn from exceptional colleagues with diverse backgrounds and experiences.
• World-class benefits: On top of a competitive salary (based on your location, experience, and skills), we provide a comprehensive benefits package to enable holistic well-being for you and your family.

Your Impact

You will design, build, and deploy robust, modular, scalable, and reproducible machine learning models tailored to various customer value management and personalization applications and you will adapt and fine-tune models to ensure they deliver significant business value and meet client-specific needs.

You will develop new machine learning models and algorithms to address emerging business challenges and opportunities in customer value management. You will also perform hyperparameter tuning, model validation, and performance optimization to enhance model accuracy and efficiency.

Lastly, you will curate, wrangle, and prepare data for use in advanced analytics models, ensuring high-quality input features.

Our current tech stack includes Python, ibis, PySpark, the and our open-source data pipelining framework, Kedro. We support deployment via Docker, Databricks, and cloud platforms such as AWS, GCP, and Azure. Our CI/CD processes utilize Github Actions, and we manage package publishing with JFrog Artifactory and internal tools. We are also exploring the integration of GenAI tools to enhance our capabilities (e.g., for automated deployment)

Your Qualifications and Skills

• Degree in Computer Science or Software Engineering, or equivalent experience preferred
• 6+ years of experience with practical knowledge of software engineering concepts and best practices in a data engineering context, including unit testing for data transformations (e.g., pandas and PySpark), data validation (e.g. Great Expectations)
• Familiarity with distributed computing frameworks (e.g., Spark, ibis), containerization (e.g., Docker), and analytics libraries (e.g. pandas, numpy, scikit-learn)
• Experience with cloud platforms (e.g. AWS, Azure, GCP) and Databricks a plus
• Experience with Git Hub
• Proficiency in writing clean, maintainable, scalable, and robust code in Python in a professional setting
• Proven experience building data pipelines for advanced analytics use cases
• Experience with Generative AI technologies, including interfacing with foundational models (e.g., OpenAI GPT-4), designing AI Agents, and prompt engineering for various content creation tasks