Inference Systems Engineer: Optimize AI Serving & Latency

adaption; San Francisco (CA); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Flexible work in Bay Area

Adaption Passport travel stipend

Lunch stipend

Medical benefits & PTO

Job summary

adaption is seeking an experienced engineer to own the cost and performance of our inference stack in the San Francisco Bay Area. You will influence throughput, latency, and reliability as workloads and hardware evolve.

You'll collaborate with the serving fleet engineers and own core levers like caching, batching, quantization, and decoding, ensuring efficient model serving without sacrificing quality.

Qualifications

• 5+ years in ML systems, inference infrastructure, or performance engineering.
• Experience with model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
• Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.

Responsibilities

• Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.
• Optimize long-context prefill and decode workloads based on real production traffic.
• Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
• Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
• Build profiling and measurement systems that show where time, memory, and compute are being spent.

Skills

Python

C++

Rust

ML systems

Performance engineering

GPU performance

Tools

vLLM

SGLang

TensorRT-LLM

Job description

adaption is seeking an experienced engineer to own the cost and performance of our inference stack in the San Francisco Bay Area. You will influence throughput, latency, and reliability as workloads and hardware evolve.

You'll collaborate with the serving fleet engineers and own core levers like caching, batching, quantization, and decoding, ensuring efficient model serving without sacrificing quality.