Inference Engineer

Adaption Labs, Inc.; San Francisco (CA); Hybrid; USD 180,000 - 230,000; Full time

Benefits offered by this job

Flexible work: In-person collaboration

Adaption Passport

Lunch Stipend

Well-Being

Job summary

Adaption Labs, Inc. is seeking an ML systems engineer to own the cost and performance of the inference stack. You will optimize caching, batching, quantization, decoding, and kernel-level tuning to improve throughput and latency while preserving model quality.

You will collaborate with the serving fleet engineers and tackle real production workloads, focusing on cost-efficiency, tail latency, and reliable delivery across changing workloads and hardware. Bay Area presence required.

Qualifications

• 5+ years in ML systems, inference infrastructure, or performance engineering with measurable improvements in cost or latency.
• Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
• Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
• Strong Python skills and proficiency in C++, Rust, or another systems language.
• Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.

Responsibilities

• Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.
• Optimize long-context prefill and decode workloads based on real production traffic.
• Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
• Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
• Build profiling and measurement systems that show where time, memory, and compute are being spent.

Skills

ML systems

Python

Performance engineering

Tools

vLLM

SGLang

TensorRT-LLM

CUDA

Job description

The role

You'll own the cost and performance of our inference stack. Your work will determine how efficiently we serve models as workloads, traffic, and hardware change.

You'll work closely with the engineers operating the serving fleet while owning the core performance levers: caching, batching, quantization, decoding, and kernel-level optimization. Success means improving throughput and latency without compromising reliability or model quality.

Responsibilities

• Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.

• Optimize long-context prefill and decode workloads based on real production traffic.

• Tune routing between our infrastructure and external providers based on cost, capacity, and performance.

• Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.

• Build profiling and measurement systems that show where time, memory, and compute are being spent.

Qualifications

• 5+ years in ML systems, inference infrastructure, or performance engineering, with measurable improvements in cost or latency.

• Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.

• Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.

• Strong Python skills and proficiency in C++, Rust, or another systems language.

• Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.

About us

Most AI is frozen in place - it doesn't adapt to the world. We think that's backwards. Our mandate is to build efficient intelligence that evolves in real-time. Our vision is AI systems that are flexible, personalized, and accessible to everyone. We believe efficiency is what makes this possible - it's how we expand access and ensure innovation benefits the many, not the few. We believe in talent density: bringing together the best and most driven individuals to push the boundaries of continual adaptation. We're looking for builders and creative thinkers ready to shape the next era of intelligence.

Benefits

• Flexible work: In-person collaboration in the Bay Area, a distributed global-first team, and team offsites.

• Adaption Passport: Annual travel stipend to explore a country you've never visited. We're building intelligence that evolves alongside you, so we encourage you to keep expanding your horizons.

• Lunch Stipend: Weekly meal allowance for take-out or grocery delivery.

• Well-Being: Comprehensive medical benefits and generous paid time off.