Data Engineering Assoc Principal Engineer

PepsiCo; Hyderabad; On-site; INR 3,000,000 - 6,000,000; Full time

Job summary

PepsiCo Inc. is seeking a data engineering lead to architect and guide large-scale data products across revenue, supply chain, manufacturing and logistics.

You will lead a team of data engineers to build pipelines, store data in the PepsiCo Data Lake, and enable analytics, visualization, machine learning, and product development efforts across the company. The role emphasizes strong technical leadership, architectural maturity, and collaboration with process owners, product owners and business

Qualifications

• 10+ years in technology with 5+ years hands-on data engineering and architecture.
• 4+ years with Data Lake, Warehousing, and Data Analytics tools.
• 4+ years in Python, PySpark, Scala; SQL optimization.
• 4+ years in cloud data engineering in Azure/AWS.
• Azure cloud services expertise; Azure Certification is a plus.
• Experience integrating multi-cloud and on-premises technologies.
• Data modeling, data warehousing, high-volume ETL/ELT pipelines.
• Experience with Databricks, Redshift, Synapse or Snowflake.
• Running and scaling apps on cloud infrastructure and Kubernetes.
• Experience with GitHub, CI/CD tooling.
• Azure Data Factory, Azure Databricks and Azure ML tools.
• Experience in retail or supply chain domains is a plus.
• Familiarity with Power BI.

Responsibilities

• Technical lead for a team of data engineers, overseeing design, development and mentoring.
• Act as a subject matter expert across various digital projects.
• Collaborate with internal and external partners to structure data using unified taxonomies.
• Build and scale data pipelines from diverse sources to support product launches and analytics.
• Develop automation and monitoring frameworks for data pipeline quality and KPIs.
• Implement best practices around system integration, security, performance and data management.
• Enable business value through data, data science and BI adoption.
• Work with data science/product teams on solutioning and POC discussions.
• Evolve data platform architecture with enterprise architects and partners.
• Productionize data science models and define SLA for data products.

Skills

Data engineering

Cloud platforms

Python

PySpark

Scala

SQL optimization

Data modeling

Leadership

Communication

DevOps/DataOps

Education

B.Tech / BE in CS or related

Tools

Databricks

Azure Databricks

Azure

AWS

SQL

PowerBI

CI/CD

Kubernetes

Azure Data Factory

Job description

Overview

As a data engineering lead, you will be the key technical expert overseeing PepsiCo's data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business. You'll be empowered to create & lead a strong team of data engineers who build data pipelines into various source systems, rest data on the PepsiCo Data Lake, and enable exploration and access for analytics, visualization, machine learning, and product development efforts across the company. As a member of the data engineering team, you will help lead the development of very large and complex data applications into public cloud environments directly impacting the design, architecture, and implementation of PepsiCo's flagship data products around topics like revenue management, supply chain, manufacturing, and logistics. You will work closely with process owners, product owners and business users.

Responsibilities

• Technical lead for a team of data engineers, managing processes and their flow of work, vetting their designs, and mentoring them to realize their full potential.
• Act as a subject matter expert across different digital projects.
• Overseework with internal clients and external partners to structure and store data into unified taxonomies and link them together with standard identifiers.
• Manage and scale data pipelines from internal and external data sources to support new product launches and drive data quality across data products.
• Build and own the automation and monitoring frameworks that captures metrics and operational KPIs for data pipeline quality and performance.
• Responsible for implementing best practices around systems integration, security, performance, and data management.
• Empower the business by creating value through the increased adoption of data, data science and business intelligence landscape.
• Collaborate with internal clients (data science and product teams) to drive solutioning and POC discussions.
• Evolve the architectural capabilities and maturity of the data platform by engaging with enterprise architects and strategic internal and external partners.
• Develop and optimize procedures to productionize data science models.
• Define and manage SLA’s for data products and processes running in production.
• Support large-scale experimentation done by data scientists.
• Prototype new approaches and build solutions at scale.
• Research in state-of-the-art methodologies.
• Create documentation for learnings and knowledge transfer.
• Create and audit reusable packages or libraries.

Qualifications

• 10+ years of overall technology experience that includes at least 5+ years of hands‑on software development, data engineering, and systems architecture.
• 4+ years of experience with Data Lake Infrastructure, Data Warehousing, and Data Analytics tools.
• 4+ years of experience in SQL optimization and performance tuning, and development experience in programming languages like Python, PySpark, Scala etc)).
• 4+ years in cloud data engineering experience in Azure/AWS.
• Fluent with Azure cloud services. Azure Certification is a plus.
• Fluent in integrating LLM APIs into data applications and orchestrating Agentic AI workflows into full stack apps (backend and frontend).
• Experience with integration of multi cloud services with on‑premises technologies.
• Experience with data modelling, data warehousing, and building high‑volume ETL/ELT pipelines.
• Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.
• Experience with at least one MPP database technology such as Databricks, Redshift, Synapse or Snowflake.
• Experience with running and scaling applications on the cloud infrastructure and containerized services like Kubernetes.
• Experience with version control systems like Github and deployment & CI tools.
• Experience with Azure Data Factory, Azure Databricks and Azure Machine learning tools.
• Experience with building solutions in the retail or in the supply chain space is a plus.
• Understanding of metadata management, data lineage, and data glossaries is a plus.
• Working knowledge of agile development, including DevOps and DataOps concepts.
• Familiarity with business intelligence tools (such as PowerBI).

Skills, Abilities, Knowledge:

• Excellent communication skills, both verbal and written, along with the ability to influence and demonstrate confidence in communications with senior level management.
• Proven track record of leading, mentoring data teams.
• Strong change manager. Comfortable with change, especially that which arises through company growth.
• Ability to understand and translate business requirements into data and technical requirements.
• High degree of organization and ability to manage multiple, competing projects and priorities simultaneously.
• Positive and flexible attitude to enable adjusting to different needs in an ever‑changing environment.
• Strong leadership, organizational and interpersonal skills; comfortable managing trade‑offs.
• Foster a team culture of accountability, communication, and self‑management.
• Proactively drives impact and engagement while bringing others along.
• Consistently attain/exceed individual and team goals.
• Ability to lead others without direct authority in a matrixed environment.