Principal AI Data Platform Engineer

Yahoo Holdings Inc.; Nashville (TN); Hybrid; USD 144,000 - 299,000; Full time

Job summary

Yahoo is seeking a Principal Engineer (IC5) to lead architecture and development of AI-powered data intelligence platforms at Yahoo scale. You will shape technical direction for enterprise AI systems and build reusable platform capabilities adopted across engineering, analytics, governance, and business units.

You will work at the intersection of large-scale distributed data systems, knowledge, and AI techniques, defining how AI reasons over Yahoo's data ecosystem to detect anomalies, diagnose

Qualifications

• 10+ years building distributed systems and large-scale data platforms.
• Degree in CS, AI/ML, Engineering, Math, or related field.
• Deep Python expertise and strong software engineering.
• Production experience with batch and streaming processing (Kafka, or equivalent).
• Shipped and operated RAG applications in production; search, and reranking.
• Strong communication and technical leadership across cross-functional stakeholders.

Responsibilities

• Define technical vision and architecture for Yahoo's AI-powered data intelligence platform.
• Architect petabyte-scale batch and streaming systems with Beam, Dataflow, Spark, and Kafka.
• Build production RAG and multi-agent systems: semantic metadata integration, agent orchestration.
• Establish evaluation frameworks - retrieval quality, agent accuracy, hallucination rate, operational safety - with automated benchmarking, so we ship AI we can trust.
• Use AI development tools to accelerate design, implement the bar for how the org validates AI-generated code and agent plans.
• Drive adoption of RAG and agentic patterns across engineers.
• Partner with product, governance, infrastructure, and exit-form priorities.

Skills

Distributed systems

Python

Leadership

Communication

Engineering discipline

Education

Degree in CS/AI/Engineering

Tools

Kafka

Job description

Yahoo is seeking a Principal Engineer (IC5) to lead architecture and development of AI-powered data intelligence platforms at Yahoo scale. You will shape technical direction for enterprise AI systems and build reusable platform capabilities adopted across engineering, analytics, governance, and business units.

You will work at the intersection of large-scale distributed data systems, knowledge, and AI techniques, defining how AI reasons over Yahoo's data ecosystem to detect anomalies, diagnose