Staff Engineer: Scalable Model Serving & Inference

Databricks; California (MO); On-site; USD 192,000 - 260,000; Full time

Job summary

Databricks is seeking a Staff Engineer to shape the Model Serving platform, designing scalable systems and APIs for high-throughput, low-latency inference on CPU and GPU. You will influence architectural direction and collaborate across platform, product, infrastructure, and research teams to deliver a world-class serving platform.

You will lead architectural decisions, drive performance optimizations, and mentor engineers, while ensuring operational excellence and adherence to best practices in

Qualifications

• 10+ years of experience building and operating large-scale distributed systems.
• Deep expertise in model serving, inference systems, and related infrastructure (routing, scheduling, autoscaling, observability).
• Strong foundation in algorithms, data structures, and system design for low-latency serving systems.

Responsibilities

• Design and implement core systems and APIs powering Databricks Model Serving, ensuring scalability and reliability.
• Define the technical roadmap and long-term architecture for serving workloads.
• Drive architectural decisions to optimize performance, throughput, and autoscaling for CPU and GPU workloads.
• Contribute to components across the serving infrastructure (model containers, deployment workflows, routing, caching, observability, autoscaling).
• Collaborate with product, platform, and research teams to translate customer needs into reliable systems.
• Lead initiatives to improve latency, availability, and cost-effectiveness across serving layers.
• Establish best practices for code quality, testing, and operational readiness; mentor engineers.

Skills

Distributed systems

Model serving

Inference systems

System design

CPU/GPU inference

Architectural direction

Leadership

Mentoring

Job description

Databricks is seeking a Staff Engineer to shape the Model Serving platform, designing scalable systems and APIs for high-throughput, low-latency inference on CPU and GPU. You will influence architectural direction and collaborate across platform, product, infrastructure, and research teams to deliver a world-class serving platform.

You will lead architectural decisions, drive performance optimizations, and mentor engineers, while ensuring operational excellence and adherence to best practices in