Senior AI Retrieval Architect

Databricks; California (MO); Hybrid; USD 229,000 - 343,000; Full time

Job summary

Databricks is hiring for a senior role focused on building and scaling retrieval systems for enterprise SaaS data. You will own end-to-end retrieval, including query understanding, content extraction, indexing, hybrid retrieval, ranking, and evaluation, to support Databricks agents and human users.

The role is based in Mountain View, CA or San Francisco, CA with hybrid in-office collaboration. Ideal candidates have 10+ years in software engineering, deep IR expertise, experience with RAG

Qualifications

• 10+ years of software engineering experience.
• Deep Information Retrieval expertise: lexical retrieval, dense retrieval, hybrid retrieval, learning-to-rank.
• Hands-on experience with modern LLM-era retrieval, RAG architectures, grounding techniques.
• Experience designing agentic systems on top of retrieval with planners, multi-hop retrieval, and tool use.
• Strong grasp of relevance evaluation: nDCG, MRR, Precision@K, Recall@K; offline/online experimentation.

Responsibilities

• Build the retrieval stack from scratch for enterprise SaaS data across multiple systems.
• Create search subagents that manage context retrieval, ranking signals, and multi-hop searches.
• Index and rank across structured assets (tables, SQL queries, dashboards, code) and unstructured content (docs, wikis, tickets, chat, images).
• Connect to SaaS surface areas; build connectors and retrieval adapters for enterprise data sources.
• Define architectural direction and mentor engineers; collaborate with Research, Product, and Platform teams.

Skills

Software engineering

Information Retrieval

LLM retrieval

Agentic systems

Evaluation metrics

Structured data handling

End-to-end retrieval

Technical leadership

Job description

Databricks is hiring for a senior role focused on building and scaling retrieval systems for enterprise SaaS data. You will own end-to-end retrieval, including query understanding, content extraction, indexing, hybrid retrieval, ranking, and evaluation, to support Databricks agents and human users.

The role is based in Mountain View, CA or San Francisco, CA with hybrid in-office collaboration. Ideal candidates have 10+ years in software engineering, deep IR expertise, experience with RAG