Senior Data Engineer

Yahoo Holdings Inc.; Richardson (TX); Hybrid; USD 128,250 - 266,875; Full time

Job summary

Yahoo Mail's analytics infrastructure team seeks a Senior Data Engineer to define the global data ontology and own core datasets for Yahoo Mail at scale. You will architect and deliver high-reliability batch and streaming data pipelines powering analytics across a massive platform.

You will collaborate with Data Science, ML, and Product teams to enable data-driven decisions, implement governance, and drive performance optimizations in a hybrid work environment.

Qualifications

• BS/MS in Computer Science, Engineering, or a related technical field.
• 6+ years building scalable ETL/ELT pipelines with orchestration (Airflow/Composer/Oozie).
• Deep expertise in SQL, PySpark, or Scala.
• Experience handling multi-terabyte/petabyte datasets and large-scale challenges.
• Cloud experience (GCP/AWS/Azure) with GitOps (CI/CD, PRs).
• GDPR and data privacy compliance experience; strong communication.

Responsibilities

• Architect data ontology and lead the technical roadmap for Yahoo Mail datasets.
• Design, build, and maintain scalable batch and streaming pipelines for the data lakehouse.
• Develop self-service data tooling and AI-enabled frameworks for data products.
• Establish data governance, SLA management, and lifecycle standards.
• Optimize performance of large-scale data infrastructures and algorithms.
• Collaborate cross-functionally as a data consultant across Mail engineering teams.

Skills

SQL

PySpark

Scala

ETL/ELT pipelines

Airflow

Education

BS/MS in Computer Science, Engineering, or a relevant technical field

Tools

Airflow

Composer

Oozie

BigQuery

Dataflow

Looker

Job description

Yahoo Mail is the ultimate consumer inbox with hundreds of millions of users. It's the best way to access your email and stay organized from a computer, phone or tablet. With its beautiful design and lightning fast speed, Yahoo Mail makes reading, organizing, and sending emails easier than ever.

A Little About Us

The Mail Analytics Infrastructure & Data Engineering team at Yahoo is the backbone of Yahoo Mail's success. We build mission-critical data lakehouses, high-performance pipelines, and the analytics frameworks that power one of the world's largest communication platforms. We are currently pushing the boundaries of our data ecosystem to harness massive datasets, directly fueling the explosive growth of Yahoo's AI-driven capabilities.

A Lot About You

You are a Senior Data Engineer who thrives on complexity and scale. You don't just build pipelines; you define the data ontology for the entire Mail organization. You are a strategic thinker who can influence event instrumentation at the source while collaborating with Data Science and ML teams to deliver high-impact analytics.

Responsibilities

• Architect: Partner with Data Science, Product, and Engineering to define the global data ontology for Yahoo Mail and lead the technical roadmap for core datasets.
• Build Scalable Systems: Design, build, and maintain high-reliability batch and streaming data pipelines that populate our mission-critical data lakehouse.
• Innovate Tooling: Develop automated frameworks and self-service tools that streamline how users interact with data products across the company. Leveraging autonomous AI agents and AI Tooling.
• Data Governance: Establish standard methodologies for data operations, lifecycle management, and strict SLA management for all datasets within your ownership area.
• Optimize Performance: Improve existing large-scale data infrastructures by applying advanced algorithmic concepts to optimize code and underlying data system stacks.
• Collaborate: Act as a data consultant for complex cross-functional projects, ensuring integrated solutions across Mail engineering teams.

Qualifications

• Education: BS/MS in Computer Science, Engineering, or a relevant technical field.
• Experience: 6+ years of experience building scalable ETL/ELT pipelines using industry-standard orchestration (Airflow, Composer, or Oozie).
• Technical Mastery: Deep expertise in SQL, PySpark, or Scala.
• Scale: Proven track record of managing Multi-Terabyte/Petabyte datasets and solving large-scale challenges (e.g., skew mitigation, data sketches, and accumulation patterns).
• Cloud & DevOps: Professional experience with at least one major cloud provider (GCP, AWS, or Azure) and a strong command of GitOps workflows (CI/CD, PRs).
• Compliance: Experience working within GDPR and other data privacy frameworks.
• Soft Skills: Exceptional communication skills with the ability to prioritize tasks in a high-pressure, fast-paced environment.

Preferred Qualifications

• GCP Expertise: 3+ years of experience with Google Cloud Platform (BigQuery, Dataproc, Dataflow, Composer, Looker).
• AI/ML Alignment: Experience building data features specifically optimized for Machine Learning models and AI applications.

The material job duties and responsibilities of this role include those listed above as well as adhering to Yahoo policies ; exercising sound judgment ; working effectively, safely and inclusively with others ; exhibiting trustworthiness and meeting expectations ; and safeguarding business operations and brand integrity.

At Yahoo, we offer flexible hybrid work options that our employees love! While most roles don't require regular office attendance, you may occasionally be asked to attend in-person events or team sessions. You'll always get notice to make arrangements. Your recruiter will let you know if a specific job requires regular attendance at a Yahoo office or facility. If you have any questions about how this applies to the role, just ask the recruiter!

Yahoo is proud to be an equal opportunity workplace. All qualified applicants will receive consideration for employment without regard to, and will not be discriminated against based on age, race, gender, color, religion, national origin, sexual orientation, gender identity, veteran status, disability or any other protected category. Yahoo will consider for employment qualified applicants with criminal histories in a manner consistent with applicable law. Yahoo is dedicated to providing an accessible environment for all candidates during the application process and for employees during their employment. If you need accessibility assistance and/or a reasonable accommodation due to a disability, please submit a request via the Accommodation Request Form ( www.yahooinc.com/careers/contact-us.html ) or call +1.866.772.3182 . Requests and calls received for non-disability related issues, such as following up on an application, will not receive a response.

We believe that a diverse and inclusive workplace strengthens Yahoo and deepens our relationships. When you support everyone to be their best selves, they spark discovery, innovation and creativity. Among other efforts, our 11 employee resource groups (ERGs) enhance a culture of belonging with programs, events and fellowship that help educate, support and create a workplace where all feel welcome.

The compensation for this position ranges from $128,250.00 - $266,875.00/yr and will vary depending on factors such as your location, skills and experience.The compensation package may also include incentive compensation opportunities in the form of discretionary annual bonus or commissions. Our comprehensive benefits include healthcare, a great 401k, backup childcare, education stipends and much (much) more.