Lead Architect, AI-Driven Data Intelligence at Scale

Yahoo Holdings Inc. in; Richardson (TX); Hybrid; USD 144,000 - 299,000; Full time

Job summary

Yahoo Holdings Inc. is seeking a Principal Engineer (IC5) to lead architecture and development of AI-powered data intelligence platforms at scale.

You will shape system design across large-scale data processing, RAG, and agentic AI, partnering with product, governance, and infrastructure teams. You will establish reference architectures, build reusable platform capabilities, and drive adoption of RAG/agentic patterns while maintaining correctness, reliability, and security across Yahoo's data

Qualifications

• 10+ years building distributed systems or equivalent experience.
• Degree in CS, AI/ML, Engineering, or related field.
• Strong Python software engineering skills.
• Production experience with batch and streaming processing (Kafka).

Responsibilities

• Define technical vision and architecture for Yahoo's AI-powered data intelligence platform.
• Architect petabyte-scale batch and streaming systems with Beam, Dataflow, Spark, and Kafka.
• Build production RAG and multi-agent systems: semantic metadata integration and agent orchestration.
• Establish evaluation frameworks for retrieval quality, agent accuracy, and operational safety.
• Drive adoption of RAG and agentic patterns across engineering teams.

Skills

Distributed systems

Python

Technical leadership

Cross-functional collaboration

AI and ML concepts

Data governance awareness

Education

Degree in CS/AI/Engineering/Math

Tools

Beam

Dataflow

Spark

Kafka

BigQuery

Vertex AI

LangGraph

Job description

Yahoo Holdings Inc. is seeking a Principal Engineer (IC5) to lead architecture and development of AI-powered data intelligence platforms at scale.

You will shape system design across large-scale data processing, RAG, and agentic AI, partnering with product, governance, and infrastructure teams. You will establish reference architectures, build reusable platform capabilities, and drive adoption of RAG/agentic patterns while maintaining correctness, reliability, and security across Yahoo's data