Senior Staff Applied AI Engineer — Context Retrieval

Databricks Inc.; Mountain View (CA); On-site; USD 260,000 - 380,000; Full time

Job summary

Databricks is hiring a Senior Staff Applied AI Engineer to own context retrieval for Databricks agents across SaaS providers. This zero-to-one role focuses on building the end-to-end retrieval stack, from query understanding to evaluation, spanning structured tables, dashboards, code, docs, and more.

You will design connectors, rank content by freshness and permissions, and optimize for both LLM-driven grounding and human discovery.

Qualifications

• 10+ years of software engineering experience in retrieval or search systems.
• Significant experience building production retrieval, search, or RAG systems at scale.
• Deep Information Retrieval (IR) expertise: lexical, dense, and hybrid retrieval.
• Hands-on with modern LLM-era retrieval and RAG architectures.
• Experience designing agentic systems on top of retrieval.

Responsibilities

• Build the full retrieval stack from scratch for enterprise context retrieval.
• Retrieve across structured and unstructured data and index/rank across modalities.
• Develop connectors and adapters for SaaS data sources with freshness and permissions signals.
• Optimize retrieval for both LLMs and human users.
• Advance query understanding, decomposition, and entity resolution for agents.
• Develop content understanding pipelines and embeddings, summaries, metadata.
• Design and implement search subagents enabling multi-hop searches and grounded evidence.
• Build an evaluation flywheel with offline/online metrics and human-in-the-loop labeling.
• Define technical direction and mentor senior engineers.

Skills

Software engineering

IR expertise

BM25

Dense retrieval

Hybrid retrieval

LLM retrieval

Learning-to-rank

Multi-hop search

Agentic systems

Education

Bachelor's in Computer Science

PhD in related field

Tools

FAISS

ScaNN

Elasticsearch

OpenSearch

Lucene

Job description

Databricks is hiring a Senior Staff Applied AI Engineer to own context retrieval for Databricks agents across SaaS providers. This zero-to-one role focuses on building the end-to-end retrieval stack, from query understanding to evaluation, spanning structured tables, dashboards, code, docs, and more.

You will design connectors, rank content by freshness and permissions, and optimize for both LLM-driven grounding and human discovery.