Inference Systems Performance Engineer

Adaption Labs; San Francisco, Northern (CA, KY); Hybrid; USD 180,000 - 260,000; Full time

Benefits offered by this job

Annual travel stipend

Lunch stipend

Well-Being benefits

Job summary

Adaption Labs is seeking a performance-focused ML systems engineer to own the cost and throughput of our inference stack. You will work with the serving fleet to optimize caching, batching, quantization, decoding, and kernel-level performance as workloads evolve.

You will collaborate with engineers, tune routing to external providers, and build profiling tools to reveal time and memory hot spots. Strong Python plus C++/Rust are highly valued.

Qualifications

• 5+ years in ML systems, inference infra, or performance engineering with measurable improvements in cost or latency.
• Deep understanding of model serving: prefill, decode, batching, memory bandwidth.
• Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
• Strong Python skills and proficiency in C++, Rust, or another systems language.
• Experience with GPU performance: CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.
• Above all, we're looking for great teammates who are adaptable and bold.

Responsibilities

• Improve throughput, cost, and tail latency via KV-cache management, continuous batching, speculative decoding, and quantization.
• Optimize long-context prefill and decode workloads based on real production traffic.
• Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
• Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
• Build profiling and measurement systems that show where time, memory, and compute are being spent.

Skills

ML systems

Inference infra

Performance engineering

Python

C++/Rust

CUDA

Tools

vLLM

SGLang

TensorRT-LLM

Job description

Adaption Labs is seeking a performance-focused ML systems engineer to own the cost and throughput of our inference stack. You will work with the serving fleet to optimize caching, batching, quantization, decoding, and kernel-level performance as workloads evolve.

You will collaborate with engineers, tune routing to external providers, and build profiling tools to reveal time and memory hot spots. Strong Python plus C++/Rust are highly valued.