Principal Data Intelligence Engineer — Hybrid AI at Scale

Yahoo Holdings Inc.; Richardson (TX); Hybrid; USD 144,000 - 299,000; Full time

Benefits offered by this job

Healthcare

401k

Education stipends

Back-up childcare

Many more

Job summary

Yahoo is seeking a Principal Engineer (IC5) to lead the architecture and development of AI-powered data intelligence platforms at scale. You will define technical direction and reference architectures, building reusable capabilities adopted across engineering, analytics, governance, and business teams.

You will enable AI-assisted development, evaluation, and automation while upholding correctness, reliability, security, and safety as Yahoo expands its data-driven products and services.

Qualifications

• 10+ years building distributed systems and large-scale data platforms.
• Degree in CS, AI/ML, Engineering, Math, or related field.
• Deep Python expertise and strong software engineering skills.
• Production experience with batch and streaming processing (Kafka).
• Shipped and operated RAG applications in production; search, and reranking.
• Track record evaluating and validating AI-generated code correctness and operational safety.
• Strong communication and technical leadership across cross-functional stakeholders.

Responsibilities

• Define technical vision and architecture for Yahoo's AI-powered data intelligence platform.
• Architect petabyte-scale batch and streaming systems with Beam, Dataflow, Spark, and Kafka.
• Build production RAG and multi-agent systems: semantic metadata integration, agent orchestration.
• Establish evaluation frameworks - retrieval quality, agent accuracy, hallucination rate, operational safety - with automated benchmarking, so we ship AI we can trust.
• Use AI development tools to accelerate design, implement the bar for how the org validates AI-generated code and agent plans.
• Drive adoption of RAG and agentic patterns across engineers.
• Partner with product, governance, infrastructure, and exit-form priorities.

Skills

Distributed systems

Python

Kafka

RAG applications

Code correctness

Technical leadership

Education

CS/AI/Engineering

Tools

LangGraph

CrewAI

AutoGen

Semant

Vertex AI

BigQuery

Vector Search

Dataflow

Job description

Yahoo is seeking a Principal Engineer (IC5) to lead the architecture and development of AI-powered data intelligence platforms at scale. You will define technical direction and reference architectures, building reusable capabilities adopted across engineering, analytics, governance, and business teams.

You will enable AI-assisted development, evaluation, and automation while upholding correctness, reliability, security, and safety as Yahoo expands its data-driven products and services.