ML Inference Performance Engineer — Optimize Cost & Latency

Adaption Labs; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Flexible work

Adaption Passport

Lunch stipend

Well-Being benefits

Job summary

Adaption Labs in San Francisco is looking for a senior ML systems engineer to own the cost and performance of the inference stack. You’ll optimize caching, batching, quantization, and kernel-level performance to improve throughput and reduce latency without sacrificing model quality.

You’ll collaborate with the serving fleet engineers and tune routing, profiling, and measurement systems across production workloads. A strong background in Python/C++/Rust and GPU optimization is required.

Qualifications

• 5+ years in ML systems, inference infrastructure, or performance engineering.
• Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
• Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
• Strong Python skills and proficiency in C++, Rust, or another systems language.
• Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.

Responsibilities

• Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.
• Optimize long-context prefill and decode workloads based on real production traffic.
• Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
• Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
• Build profiling and measurement systems that show where time, memory, and compute are being spent.

Skills

ML systems

Model serving

Python

C++ / Rust

GPU performance

Tools

vLLM

SGLang

TensorRT-LLM

Job description

Adaption Labs in San Francisco is looking for a senior ML systems engineer to own the cost and performance of the inference stack. You’ll optimize caching, batching, quantization, and kernel-level performance to improve throughput and reduce latency without sacrificing model quality.

You’ll collaborate with the serving fleet engineers and tune routing, profiling, and measurement systems across production workloads. A strong background in Python/C++/Rust and GPU optimization is required.