Inference Performance Engineer: Optimize Model Serving

Adaption; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Lunch stipend

Travel stipend (Adaption Passport)

Well-being benefits

Flexible in-person collaboration

Job summary

Adaption in San Francisco Bay Area seeks a senior ML systems engineer to own cost and performance of the inference stack. You will optimize caching, batching, quantization, and decoding, while collaborating with the serving fleet to deliver scalable, low-latency model serving.

Required are 5+ years in ML systems with deep knowledge of model serving, GPU performance, and proficiency in Python and C++/Rust. You’ll work with engines like vLLM and TensorRT-LLM and contribute to profiling and

Qualifications

• 5+ years in ML systems, inference infrastructure, or performance engineering with measurable improvements in cost or latency.
• Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
• Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
• Strong Python skills and proficiency in C++, Rust, or another systems language.
• Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.

Responsibilities

• Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.
• Optimize long-context prefill and decode workloads based on real production traffic.
• Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
• Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
• Build profiling and measurement systems that show where time, memory, and compute are being spent.

Skills

ML systems

Performance engineering

Python

C++/Rust

GPU performance

Tools

vLLM

SGLang

TensorRT-LLM

CUDA

NCCL

Job description

Adaption in San Francisco Bay Area seeks a senior ML systems engineer to own cost and performance of the inference stack. You will optimize caching, batching, quantization, and decoding, while collaborating with the serving fleet to deliver scalable, low-latency model serving.

Required are 5+ years in ML systems with deep knowledge of model serving, GPU performance, and proficiency in Python and C++/Rust. You’ll work with engines like vLLM and TensorRT-LLM and contribute to profiling and