Senior Engineer, Model Serving & Low-Latency AI

Databricks; California (MO); On-site; USD 166,000 - 225,000; Full time

Job summary

Databricks is seeking a Senior Engineer to shape the Model Serving platform and its infrastructure, enabling high-throughput, low-latency inference across CPU and GPU workloads.

You will design core systems, guide architectural decisions, and collaborate with platform, product, infrastructure, and research teams to deliver scalable, reliable serving capabilities while mentoring teammates and promoting engineering excellence.

Qualifications

• 5+ years of experience building and operating large-scale distributed systems.
• Experience in model serving, inference systems, or related infrastructure (e.g., routing, scheduling, autoscaling, observability).
• Strong foundation in algorithms, data structures, and system design as applied to large-scale, low-latency serving systems.

Responsibilities

• Design and implement core systems and APIs powering Databricks Model Serving, ensuring scalability and reliability.
• Drive architectural decisions to optimize performance, throughput, and operational efficiency for CPU/GPU workloads.
• Contribute to components across the serving stack—from model container builds to routing, caching, and observability.

Skills

Distributed systems

Model serving / inference

Algorithms & data structures

Mentoring / leadership

Cross-functional collaboration

Job description

Databricks is seeking a Senior Engineer to shape the Model Serving platform and its infrastructure, enabling high-throughput, low-latency inference across CPU and GPU workloads.

You will design core systems, guide architectural decisions, and collaborate with platform, product, infrastructure, and research teams to deliver scalable, reliable serving capabilities while mentoring teammates and promoting engineering excellence.