Staff AI Research Infrastructure Engineer

Doist; San Francisco, Northern (CA, KY); Hybrid; USD 190,000 - 270,000; Full time

Job summary

Databricks is seeking a Staff Software Engineer for AI Research Infrastructure to design and run the research stack powering Databricks AI Research. You will build services that schedule, orchestrate, and observe large-scale training and inference workloads across GPUs, while ensuring reliability and security.

You will collaborate with researchers, ML engineers, and platform teams to turn experimental workloads into robust pipelines, advance tooling, and shape the future of our compute

Qualifications

• 5+ years of software engineering with large-scale distributed systems.
• Experience building/operating cluster schedulers and data pipelines.
• Familiarity with ML training/inference workflows and GPUs.

Responsibilities

• Design and implement infrastructure for large-scale experiments.
• Enable researchers to move from idea to large-scale experiments quickly with scheduling and monitoring abstractions.
• Create tooling that boosts researcher productivity (CI/testing for research code, workflows).
• Influence the long-term roadmap for research compute infrastructure.
• Mentor other engineers working on compute, infra, and AI systems.

Skills

Distributed systems

Backend services

System programming

C++

Rust

Go

Java

Scala

Education

BS/MS or PhD in Computer Science or related field

Tools

Kubernetes

Slurm

Ray

GPU clusters

Cloud platforms

HPC

Job description

Databricks is seeking a Staff Software Engineer for AI Research Infrastructure to design and run the research stack powering Databricks AI Research. You will build services that schedule, orchestrate, and observe large-scale training and inference workloads across GPUs, while ensuring reliability and security.

You will collaborate with researchers, ML engineers, and platform teams to turn experimental workloads into robust pipelines, advance tooling, and shape the future of our compute