AI Inference Throughput & Cost Engineer

adaption; Paris; Sur place; EUR 90 000 - 140 000; Plein temps

Avantages offerts par ce poste

Flexible work

Adaption Passport

Lunch Stipend

Well-Being

Résumé du poste

adaption is seeking an expert to own the cost and performance of its inference stack, driving throughput and latency improvements as workloads, traffic, and hardware evolve. You’ll partner with the serving fleet engineers to optimize caching, batching, quantization, and kernel-level tuning to achieve reliable, high-quality model deployments.

You will work with engines like vLLM, SGLang, and TensorRT-LLM while building profiling systems to illuminate time, memory, and compute usage across the

Qualifications

• 5+ years in ML systems or inference infrastructure with measurable cost/latency improvements.
• Deep understanding of model serving, including prefill, decode, batching, and memory bandwidth.
• Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
• Strong Python and systems language skills (C++, Rust or similar).
• Experience with GPU performance (CUDA, NCCL, mixed precision).

Responsabilités

• Improve throughput, cost, and tail latency via KV-cache management, continuous batching, speculative decoding, and quantization.
• Optimize long-context prefill and decode workloads based on production traffic.
• Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
• Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
• Build profiling and measurement systems that show where time, memory, and compute are being spent.

Connaissances

ML systems

Inference infrastructure

Performance engineering

Python

C++/Rust

GPU performance

Outils

vLLM

SGLang

TensorRT-LLM

CUDA

NCCL

Description du poste

adaption is seeking an expert to own the cost and performance of its inference stack, driving throughput and latency improvements as workloads, traffic, and hardware evolve. You’ll partner with the serving fleet engineers to optimize caching, batching, quantization, and kernel-level tuning to achieve reliable, high-quality model deployments.

You will work with engines like vLLM, SGLang, and TensorRT-LLM while building profiling systems to illuminate time, memory, and compute usage across the