Principal Data Intelligence Engineer — AI Platform (Hybrid)

Yahoo Holdings Inc. in; Baltimore (MD); Hybrid; USD 144,000 - 299,000; Full time

Job summary

Yahoo Holdings Inc. is hiring a Principal Engineer (IC5) to lead architecture and development of AI-powered data intelligence platforms at Yahoo scale.

You'll define technical direction across enterprise data systems, build petabyte-scale batch/streaming pipelines with Beam, Dataflow, Spark, and Kafka, and develop RAG and multi-agent capabilities with strong emphasis on correctness, reliability, and safety.

Qualifications

• 10+ years building distributed systems and large-scale data platforms.
• Degree in CS, AI/ML, Engineering, Math, or related field.
• Deep Python expertise and strong software engineering fun
• Production experience with batch and streaming processing (Kafka, or equivalent).
• Shipped and operated RAG applications in production; search, and reranking.
• Track record evaluating and validating AI-generated code correctness and operational safety.
• Strong communication and technical leadership across cross-functional stakeholders.

Responsibilities

• Define technical vision and architecture for Yahoo's AI-powered data intelligence platform.
• Architect petabyte-scale batch and streaming systems with Beam, Dataflow, Spark, and Kafka.
• Build production RAG and multi-agent systems: semantic metadata integration, agent orchestration.
• Establish evaluation frameworks - retrieval quality, agent accuracy, hallucination rate, operational safety - with automated benchmarking, so we ship AI we can trust.
• Use AI development tools to accelerate design, implement the bar for how the org validates AI-generated code and agent plans.
• Drive adoption of RAG and agentic patterns across engineers.
• Partner with product, governance, infrastructure, and exit-form priorities.

Skills

Distributed systems

Python

Technical leadership

AI/ML knowledge

Cross-functional collaboration

Education

Bachelor's degree in CS/Engineering/Math

Tools

Kafka

Beam

Dataflow

Spark

LangGraph

CrewAI

AutoGen

Semant

Job description

Yahoo Holdings Inc. is hiring a Principal Engineer (IC5) to lead architecture and development of AI-powered data intelligence platforms at Yahoo scale.

You'll define technical direction across enterprise data systems, build petabyte-scale batch/streaming pipelines with Beam, Dataflow, Spark, and Kafka, and develop RAG and multi-agent capabilities with strong emphasis on correctness, reliability, and safety.