Senior Staff Applied AI Engineer, Retrieval & Agentic IR

Databricks Inc.; San Francisco (CA); Hybrid; USD 229,000 - 343,000; Full time

Job summary

Databricks is hiring a Senior Staff Applied AI Engineer to own context retrieval for Databricks agents across SaaS providers. You will lead end-to-end retrieval, design cross-modal signals, and build agentic search subagents that reason about needed sources with strong evaluation loops.

You will work across structured and unstructured data, connect to enterprise data surfaces, and mentor engineers while shaping the platform for LLM-based agents and human discovery.

Qualifications

• 10+ years of software engineering experience building production retrieval, search, or RAG systems.
• IR expertise: lexical, dense, hybrid retrieval and learning-to-rank.
• Hands-on experience with LLM-era retrieval and RAG architectures.
• Experience designing agentic retrieval systems with multi-hop planning.
• Strong grasp of relevance evaluation and offline/online experimentation.
• Experience across structured and unstructured data indexing and ranking.

Responsibilities

• Build the full retrieval stack from scratch for end-to-end access to context.
• Retrieve across structured assets and unstructured content with modality-specific signals.
• Connect to SaaS surface areas with connectors and adapters respecting freshness and permissions.
• Optimize for both LLMs and humans with appropriate signals.
• Crack query understanding for agentic workflows with rewriting and decomposition.
• Crack content understanding at scale extracting structure, embeddings and metadata.
• Build search subagents that plan multi-hop searches and verify retrieved evidence.
• Build evaluation flywheel including offline metrics and human labeling.
• Set technical direction and grow the team.

Skills

10+ years software engineering

IR expertise

LLM-era retrieval

Agentic systems

Evaluation metrics

Structured & unstructured data

01 building

Technical leadership

Elasticsearch/OpenSearch

FAISS/ScaNN/HNSW

Tools

Elasticsearch/OpenSearch

FAISS

ScaNN

HNSW

Job description

Databricks is hiring a Senior Staff Applied AI Engineer to own context retrieval for Databricks agents across SaaS providers. You will lead end-to-end retrieval, design cross-modal signals, and build agentic search subagents that reason about needed sources with strong evaluation loops.

You will work across structured and unstructured data, connect to enterprise data surfaces, and mentor engineers while shaping the platform for LLM-based agents and human discovery.