Staff Software Engineer: Foundation Model Inference at Scale

Databricks; San Francisco (CA); On-site; USD 190,000 - 265,000; Full time

Job summary

Databricks seeks highly autonomous backend/infrastructure engineers to power enterprise-scale model inference on the Mosaic AI platform. You will build and optimize LLM infrastructure, serving OpenAI, Anthropic, Gemini and self-hosted models, tackling reliability and latency at scale.

Collaborate with platform, infra, and ML teams to deliver end-to-end experiences, influence developer workflows, and drive observability.

Qualifications

• 8+ years of experience in backend or infrastructure engineering.
• Experience with distributed systems, scalable APIs, or cloud-native infrastructure.
• Experience with real-time serving, ML infrastructure, or GPU orchestration.
• Familiarity with service-oriented architecture, deployment pipelines, and system observability.

Responsibilities

• Build LLM infrastructure powering large-scale inference workloads for customers through partner models (OpenAI, Anthropic, Gemini) and self-hosted models (Qwen, GPT-OSS, Llama).
• Improve reliability, latency, and efficiency of distributed AI workloads.
• Collaborate with platform, infra, and ML teams to deliver seamless end-to-end experiences.
• Shape how developers and data scientists build and interact with AI on Databricks.

Skills

Backend engineering

Distributed systems

Cloud-native infrastructure

Real-time serving

GPU orchestration

Observability

Tools

SageMaker

Vertex AI

Azure ML

MLflow

PyTorch

Ray

vLLM

SGLang

Job description

Databricks seeks highly autonomous backend/infrastructure engineers to power enterprise-scale model inference on the Mosaic AI platform. You will build and optimize LLM infrastructure, serving OpenAI, Anthropic, Gemini and self-hosted models, tackling reliability and latency at scale.

Collaborate with platform, infra, and ML teams to deliver end-to-end experiences, influence developer workflows, and drive observability.