Lead Data Engineer - T&T

McCormick & Company, Incorporated; Gurugram District; Hybrid; INR 4,000,000 - 6,000,000; Full time

Benefits offered by this job

Career growth opportunities

Flexibility and support for life-stage

Job summary

McCormick & Company, Incorporated is seeking a Lead Data Engineer to drive data product delivery from multiple source systems, enabling AI, analytics, and data-driven decisions for business units. You will design and scale ELT/ETL pipelines, ensure data quality and security, and collaborate with data science teams to optimize performance and cost.

The role emphasizes building robust pipelines using PySpark, Azure Synapse, APIs, and SQL, with a focus on data modeling, automation, and ELT/ETL

Qualifications

• Bachelor's degree in Mathematics, Statistics, Computer Science, Data Analytics/Science, or related field.
• Microsoft Certified: Azure Data Engineer (DP203) or Fabric Data Engineer Associate or related cloud technologies, Fabric IQ/Databricks certifications a plus
• 8+ years of data engineering experience.
• Demonstrated ability coding in one or more languages (PySpark preferred).
• Experience with building data pipelines.
• Experience with knowledge graphs a plus.
• Demonstrated ability to manage multiple priorities simultaneously.
• Demonstrated ownership of production-grade pipelines.

Responsibilities

• Deliver and support data for existing analytics solutions, tooling, and data products.
• Research new features and implement automations.
• Support business users, Data Scientists and Data Analysts to convert business requirements into data products and data models.
• Build and maintain scalable data pipelines from source systems.
• Ensure availability, reliability, and performance of data products by integrating data from multiple sources.

Skills

PySpark

ETL development

Data modeling

Education

Bachelor's degree in Mathematics/Statistics/CS/Data Analytics

Azure DP-203/Data Engineer certification

Fabric Data Engineer Associate

Tools

Azure Synapse

SQL

Databricks

Job description

Select how often (in days) to receive an alert:

You may know McCormick as a leader in herbs, spices, seasonings, and condiments – and we’re only getting started. At McCormick, we’re always looking for new people to bring their unique flavor to our team.

McCormick employees – all 14,000 of us across the world – are what makes this company a great place to work.

What We Bring To The Table:

The best people deserve the best rewards. In addition to the benefits you’d expect from a global leader (401k, health insurance, paid time off, etc.) we also offer:

• Career growth opportunities
• Flexibility and Support for Diverse Life Stages and Choices

As a Lead Data Engineer at McCormick, you will play a pivotal role in the build and delivery of data products from simple to complex and supporting McCormick business units with their data and analytics needs.

Your responsibilities will include delivering and supporting data for existing analytics solutions, tooling, and solutions, researching new features and implementing automations. You will support business users, Data Scientists and Data Analysts to convert business expectations into data products and data models usable by business to deliver AI, analysis, reporting, and data-driven recommendations to stakeholders and executives.

This role will be accountable for building and maintaining scalable data pipelines from source systems. The Data Engineer will ensure the availability, reliability, and performance of data products by integrating raw data from various sources. Key responsibilities include data modeling, ETL (Extract, Transform, Load) development, and ensuring data quality and security. This role will be accountable for data coming in from 5-10+ source systems.

Design and Execute

• Partner with data product managers to gather and deliver data pipelines.
• Design ETL solutions including data quality, data security, and data pipeline resiliency.
• Execute ETL solutions including data security, data quality and performance requirements.

Data Extraction, Load and Transformation

• Design, build and implement ELT pipelines to efficiently ingest and transform data from a wide variety of data sources and deliver datasets that meet business requirements.
• Optimize performance for large datasets and data workflows for performance, scalability, and reliability to support business needs.
• Develop and maintain scalable data pipelines leveraging Azure Synapse, PySpark, APIs, and SQL & performing advanced data cleaning, transformation, and manipulation to ensure high-quality, and reliable data flows.
• Implement CI/CD processes to streamline and automate data pipelines deployment
• Apply data validation frameworks (Great Expectations, Fabric-native tools) to maintain accuracy
• Utilize partitioning, indexing, clustering strategies to enhance query performance

Process Improvement, Performance and Cost optimization tuning

• Collaborate with Data Science, AI, and Data product teams to optimize performance and cost effectiveness of their solutions.
• Identify and support the design of internal process improvements, including automating manual processes, optimizing data product delivery, and redesigning solutions for enhanced scalability.
• Implement solution adjustments to improve performance and cost-effectiveness of data products.

Issue Resolution and Support

• Monitor and troubleshoot the data pipelines proactively, which includes leading the support of data-related product pipeline issues to resolve data errors.
• Provide expert-level support and guidance to data teams across the Enterprise.

Desired Candidate Profile:

• Bachelor’s degree in Mathematics, Statistics, Computer Science, Data Analytics/Science, or related field
• Microsoft Certified: Azure Data Engineer (DP203)
  or Microsoft Certified: Fabric Data Engineer Associate or related cloud technologies, Fabric IQ/Databricks certifications a plus
• 8+ yearsof data engineering experience.
• Demonstrated ability coding in one or more languages (PySpark preferred).
• Experience with building data pipelines.
• Experience with knowledge graphs a plus.
• Demonstrated ability to manage multiple priorities simultaneously.
• Demonstrated ownership of production-grade pipelines.

As a general policy, McCormick does not offer employment visa sponsorships upon hire or in the future.

McCormick & Company is an equal opportunity/affirmative action employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected veteran status, age, or any other characteristic protected by law.

WHY WORK AT MCCORMICK?

As a McCormick employee you’ll be empowered to focus on more than your individual responsibilities. You’ll have the opportunity to be part of something bigger than yourself—to have a say in where the company is going and how it’s growing.

Between our passion for flavor, our 130-year history of leadership and integrity, the competitive and comprehensive benefits we offer, and our culture, which is built on respect and opportunities for growth, there are many reasons to join us at McCormick.