Lead Data Engineer

PepsiCo; Ciudad de México; Híbrido; MXN 600,000 - 1,000,000; Jornada completa

Descripción de la vacante

An established industry player is seeking a Lead Data Engineer to join their dynamic team. In this pivotal role, you will drive the development and operation of innovative data products, overseeing the creation of robust data pipelines and ensuring seamless integration across various platforms. With a focus on scalability and performance, you will collaborate with cross-functional teams to implement cutting-edge solutions that enhance business operations. This is an exciting opportunity to make a significant impact in a supportive and inclusive environment, where your expertise will be valued and your contributions celebrated. If you are passionate about data engineering and eager to shape the future of data-driven decision-making, this role is for you.

Servicios

Flexible working hours

Recognition programs

Financial wellness programs

Scholarship programs for children

Wellness Line

Discounts and agreements

Specialized training

Opportunities for self-learning

Development programs

Formación

• 10+ years in IT with strong data engineering skills.
• Experience with Azure and building data pipelines.

Responsabilidades

• Lead delivery of data engineering projects on time and budget.
• Design scalable data solutions and manage data pipelines.

Conocimientos

SQL

Python

Data Engineering

Cloud Technologies (Azure, AWS)

Spark Optimization

Data Modeling

Data Warehousing

Communication Skills

Educación

Bachelor’s degree in Computer Science

Master’s degree (preferred)

Herramientas

Azure Data Lake

Azure Data Factory

Azure Data Bricks

Hadoop

Redshift

Descripción del empleo

Overview

We Are PepsiCo

Join PepsiCo and Dare for Better! We are the perfect place for curious people, thinkers and change agents. From leadership to front lines, we're excited about the future and working together to make the world a better place.

Being part of PepsiCo means being part of one of the largest food and beverage companies in the world, with our iconic brands consumed more than a billion times a day in more than 200 countries.

Our product portfolio, which includes 22 of the world's most iconic brands, such as Sabritas, Gamesa, Quaker, Pepsi, Gatorade and Sonrics, has been a part of Mexican homes for more than 116 years.

A career at PepsiCo means working in a culture where all people are welcome. Here, you can dare to be you. No matter who you are, where you're from, or who you love, you can always influence the people around you and make a positive impact in the world.

Know more: PepsiCoJobs

Join PepsiCo, dare for better.

Responsibilities

The Opportunity

As a member of the data engineering team, you will be the key technical expert developing and overseeing PepsiCo's data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business. You'll be an empowered member of a team of data engineers who build data pipelines into various source systems, rest data on the PepsiCo Data Lake, and enable exploration and access for analytics, visualization, machine learning, and product development efforts across the company. You will work closely with process owners, product owners and business users. You'll be working in a hybrid environment with in-house, on-premise data sources as well as cloud and remote systems.

Your Impact

As Lead Data Engineer your scope would consist of:

• Drive and lead delivery of key projects within time and budget
• Drive and lead solution design and build to ensure scalability, performance, and reuse
• Ability to recommend and drive consensus around preferred data integration/engineering approaches
• Ability to anticipate data bottlenecks (latency, quality, speed) and recommend appropriate remediation strategies
• Ensure on time and on budget delivery which satisfies project requirements, while adhering to enterprise architecture standards.
• Facilitate work intake, prioritization and release timing, balancing demand and available resources. Ensure tactical initiatives are aligned with the strategic vision and business needs.
• Ensure sustainability of live pipelines in production environment
• Hands on experience of implementing and designing data engineering workloads using Spark, Databricks, or similar modern data processing technology
• Work with product owners, scrum masters and technical committee to define the 3 months road-map for each program increment (sprint wise)
• Manage and scale data pipelines responsible for ingestion and data transformation.
• Evolve the architectural capabilities and maturity of the data platform by engaging with enterprise architects and strategic internal and external partners.
• Prototype new approaches and build solutions at scale.
• Research in state-of-the-art methodologies.
• Create documentation for learnings and knowledge transfer.
• Collaborate with internal clients (data science and product teams) to drive solutioning and POC discussions.

Qualifications

¿Who Are We Looking For?

Required Skill-set:

• Bachelor’s degree in Computer Science, MIS, Business Management, or related field
• 10 + years’ experience in Information Technology
• 4 + years of Azure, AWS and Cloud technologies
• 6+ years of experience of writing complex SQL queries in DWH or Datalake environment
• 5+ years of experience with a programming language (i.e., python, java or scala) preferably python.
• Experience of building frameworks for different processes such as data ingestion and dataops
• Good written and verbal communication skills along with collaboration and listening skills
• Well versed with Spark optimization techniques
• Experience dealing with multiple vendors as necessary.
• Hands on experience of writing complex SQL queries
• Big Data (Hadoop, HBase, MapReduce, Hive, HDFS etc.), Spark/PySpark
• Sound skills and hands on experience with Azure Data Lake, Azure Data Factory, Azure Data Bricks, Azure Synapse Analytics, Azure Storage Explorer
• Proficient in creating Data Factory pipelines for on-cloud ETL processing; copy activity, custom Azure development etc.
• Experience with data modeling, data warehousing, and building high-volume ETL/ELT pipelines.
• Experience with data profiling and data quality tools
• Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.
• Experience with at least one MPP database technology such as Redshift, Synapse or SnowFlake.

If this is an opportunity that interests you, we encourage you to apply even if you do not meet 100% of the requirements.

What can you expect from us:

• Opportunities to learn and develop every day through a wide range of programs.
• Internal digital platforms that promote self-learning.
• Development programs according to Leadership skills.
• Specialized training according to the role.
• Learning experiences with internal and external providers.
• We love to celebrate success, which is why we have recognition programs for seniority, behavior, leadership, moments of life, among others.
• Financial wellness programs that will help you reach your goals in all stages of life.
• A flexibility program that will allow you to balance your personal and work life, adapting your working day to your lifestyle.
• And because your family is also important to us, they can also enjoy benefits such as our Wellness Line, thousands of Agreements and Discounts, Scholarship programs for your children, Aid Plans for different moments of life, among others.

We are an equal opportunity employer and value diversity at our company. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We respect and value diversity as a work force and innovation for the organization.