Inference Performance Engineer — Scale AI Serving

Adaption Labs; United States; Hybrid; USD 140,000 - 210,000; Full time

Benefits offered by this job

Flexible work

Adaption Passport travel stipend

Lunch stipend

Well-being benefits

Job summary

Adaption Labs seeks an engineer to own the cost and performance of our inference stack. You will shape how we serve models as workloads shift, focusing on throughput, latency, and reliability.

You will collaborate with the serving fleet engineers, tuning caching, batching, quantization, decoding, and kernel-level optimizations to maximize efficiency without compromising model quality.

Qualifications

• 5+ years in ML systems, inference infrastructure, or performance engineering with measurable improvements in cost or latency.
• Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
• Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
• Strong Python skills and proficiency in C++, Rust, or another systems language.
• Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.

Responsibilities

• Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.
• Optimize long-context prefill and decode workloads based on real production traffic.
• Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
• Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
• Build profiling and measurement systems that show where time, memory, and compute are being spent.

Skills

ML systems

Performance engineering

Serving engines

Python/C++/Rust

GPU performance

Job description

Adaption Labs seeks an engineer to own the cost and performance of our inference stack. You will shape how we serve models as workloads shift, focusing on throughput, latency, and reliability.

You will collaborate with the serving fleet engineers, tuning caching, batching, quantization, decoding, and kernel-level optimizations to maximize efficiency without compromising model quality.