Staff Backend Engineer, AI Model Serving Platform

Neon; San Francisco (CA); On-site; USD 192,000 - 260,000; Full time

Job summary

Databricks is seeking an accomplished Staff Engineer to shape the Model Serving experience and the foundational infrastructure. You will design scalable, low-latency inference systems for CPU and GPU workloads, influencing architectural direction and collaborating with platform, product, and research teams to deliver a world-class serving platform.

You will lead architectural decisions, drive performance optimizations, and mentor engineers while aligning technical efforts with Databricks’

Qualifications

• 10+ years of experience building and operating large-scale distributed systems.
• Deep expertise in model serving, inference systems, and related infrastructure.
• Strong foundation in algorithms, data structures, and system design for low-latency serving.
• Proven ability to deliver technically complex, high-impact initiatives.
• Experience leading architecture for large-scale CPU/GPU inference systems.
• Strong communication skills and ability to collaborate across teams in fast-moving environments.
• Strategic and product-oriented mindset with ability to align technical execution with long-term vision.
• Passion for mentoring, growing engineers, and fostering technical excellence.

Responsibilities

• Design and implement core systems and APIs powering Databricks Model Serving, ensuring scalability and reliability.
• Partner with product and engineering leadership to define the technical roadmap and long-term architecture for serving workloads.
• Drive architectural decisions to optimize performance, throughput, autoscaling, and cost-efficiency for CPU/GPU workloads.
• Contribute to components across serving infrastructure—from model container builds to routing, caching, observability, and autoscaling.
• Collaborate with product, platform, and research teams to translate customer needs into reliable systems.
• Lead technical initiatives to improve latency, availability, and cost-effectiveness across serving layers.
• Establish best practices for code quality, testing, and operational readiness; mentor engineers.
• Represent the team in cross-organizational technical discussions and influence Databricks’ AI platform strategy.

Skills

Distributed systems

Model serving

CPU/GPU inference

System design

Leadership

Communication

Mentoring

Product strategy

Job description

Databricks is seeking an accomplished Staff Engineer to shape the Model Serving experience and the foundational infrastructure. You will design scalable, low-latency inference systems for CPU and GPU workloads, influencing architectural direction and collaborating with platform, product, and research teams to deliver a world-class serving platform.

You will lead architectural decisions, drive performance optimizations, and mentor engineers while aligning technical efforts with Databricks’