Inference Engineer: Maximize Throughput, Cut Latency & Cost

adaption; San Francisco (CA); Presencial; USD 150.000 - 230.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Flexible work: In-person Bay Area coll

Adaption Passport: travel stipend

Lunch Stipend: weekly meal allowance

Well-Being: medical benefits & PTO

Descripción de la vacante

adaption in San Francisco seeks an experienced ML systems engineer to own cost, performance of our inference stack and drive throughput, latency, and reliability as workloads and hardware evolve.

You will optimize caching, batching, quantization, decoding, and kernel-level tuning, collaborating with the serving fleet and integrating engines like vLLM, SGLang, and TensorRT-LLM to push efficiency without sacrificing model quality.

Formación

• 5+ years in ML systems, inference infrastructure, or performance engineering with measurable improvements.
• Deep understanding of model serving, prefill, decode, memory bandwidth, batching, and concurrency.
• Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
• Strong Python skills and proficiency in C++, Rust, or another systems language.
• Experience with GPU performance (CUDA, NCCL, mixed precision, memory layouts).

Responsabilidades

• Improve throughput, cost, and tail latency via KV-cache, continuous batching, speculative decoding, and quantization.
• Optimize long-context prefill and decode workloads based on production traffic.
• Tune routing between infrastructure and external providers by cost, capacity, and performance.
• Work within serving engines like vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
• Build profiling and measurement systems to show time, memory, and compute usage.

Conocimientos

ML systems

Performance engineering

Python

C++ / Rust

CUDA / GPU perf

Herramientas

vLLM

SGLang

TensorRT-LLM

Descripción del empleo

adaption in San Francisco seeks an experienced ML systems engineer to own cost, performance of our inference stack and drive throughput, latency, and reliability as workloads and hardware evolve.

You will optimize caching, batching, quantization, decoding, and kernel-level tuning, collaborating with the serving fleet and integrating engines like vLLM, SGLang, and TensorRT-LLM to push efficiency without sacrificing model quality.