Senior Platform Engineer, AI Model Serving

Databricks; United States; Remote; USD 166,000 - 225,000; Full time

Job summary

Databricks is seeking a Senior Engineer for its Model Serving group. You will design and build high-throughput, low-latency inference systems across CPU and GPU workloads, and shape the platform's core serving infrastructure in collaboration with product, platform, infrastructure, and research teams.

You will drive architectural decisions, implement scalable APIs, mentor peers, establish best practices, and ensure reliability, observability, and cost efficiency for enterprise deployments.

Qualifications

• 5+ years building and operating large-scale distributed systems.
• Experience in model serving, inference systems, and autoscaling.
• Strong foundation in algorithms, data structures, and system design.
• Proven ability to deliver technically complex, high-impact initiatives.
• Experience building architecture for large-scale, performance-sensitive CPU/GPU inference systems.
• Strong communication skills and ability to collaborate across teams.
• Customer-focused mindset with the ability to align implementation details with product goals.
• Passion for mentoring and growing engineers.

Responsibilities

• Design and implement core systems and APIs powering Databricks Model Serving.
• Drive architectural decisions to optimize performance, throughput, autoscaling, and efficiency.
• Contribute to components across serving infrastructure—model containers, deployment workflows, routing, caching, observability, and autoscaling.
• Collaborate with product, platform, and research teams to translate customer needs into reliable systems.
• Lead technical initiatives to improve latency, availability, and cost-effectiveness.
• Establish best practices for code quality, testing, and operational readiness; mentor engineers.

Skills

Distributed systems

Model serving / inference

Algorithms & data structures

High-impact delivery

CPU/GPU inference architecture

Communication across teams

Customer-focused mindset

Mentoring engineers

Job description

Databricks is seeking a Senior Engineer for its Model Serving group. You will design and build high-throughput, low-latency inference systems across CPU and GPU workloads, and shape the platform's core serving infrastructure in collaboration with product, platform, infrastructure, and research teams.

You will drive architectural decisions, implement scalable APIs, mentor peers, establish best practices, and ensure reliability, observability, and cost efficiency for enterprise deployments.