Lead Data Intelligence Platform Architect

Yahoo Holdings Inc.; Mountain View (CA); Hybrid; USD 144,000 - 299,000; Full time

Benefits offered by this job

Flexible hybrid work

Healthcare

401k

Job summary

Yahoo Holdings Inc. seeks a Principal Engineer (IC5) to lead the architecture and development of AI-powered data intelligence platforms at Yahoo scale.

You will shape how AI reasons over Yahoo's data ecosystem to help teams detect anomalies, diagnose incidents, and optimize infrastructure. The role requires establishing direction, building reusable platform capabilities, and ensuring rigorous correctness and security.

Qualifications

• 10+ years building distributed systems and large-scale data platforms.
• Degree in CS, AI/ML, Engineering, Math, or related field.
• Deep Python expertise and strong software engineering skills.
• Production experience with batch and streaming processing (Kafka).
• Shipped and operated RAG applications in production with AI safety.
• Strong cross-functional communication and leadership.

Responsibilities

• Define technical vision and architecture for Yahoo's AI-powered data platform.
• Architect petabyte-scale batch and streaming systems (Beam, Dataflow, Spark, Kafka).
• Build production RAG and multi-agent systems with semantic metadata integration.
• Establish evaluation frameworks for retrieval quality, agent accuracy, and safety.
• Accelerate design using AI tools while ensuring correctness and security.
• Drive adoption of RAG and agentic patterns across teams.

Skills

Distributed systems leadership

Python expertise

Production data processing

AI/ML knowledge

Code correctness & safety

Technical leadership

Education

CS / AI / Engineering degree

Math or related field

Tools

Kafka

Beam / Dataflow

BigQuery / Vertex AI

Vector search / RAG tooling

Job description

Yahoo Holdings Inc. seeks a Principal Engineer (IC5) to lead the architecture and development of AI-powered data intelligence platforms at Yahoo scale.

You will shape how AI reasons over Yahoo's data ecosystem to help teams detect anomalies, diagnose incidents, and optimize infrastructure. The role requires establishing direction, building reusable platform capabilities, and ensuring rigorous correctness and security.