Inference Performance Engineer: AI Serving at Scale

adaption; United States; Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Flexible in-person collaboration in BA

Adaption Passport travel stipend

Lunch stipend

Well-being: medical benefits and PTO

Job summary

Adaption is seeking an experienced engineer to own the cost and performance of our inference stack in a production environment. You will influence batching, caching, quantization, and kernel-level optimization while coordinating with the serving fleet to meet throughput and latency targets.

You will optimize prefill/decode workloads, route performance between internal and external providers, and build profiling tools.

Qualifications

• 5+ years in ML systems, inference infrastructure, or performance engineering with measurable improvements in cost or latency.
• Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
• Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
• Strong Python skills and proficiency in C++, Rust, or another systems language.
• Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.

Responsibilities

• Own throughput, cost, and tail latency improvements through KV-cache management, batching, quantization, and decoding.
• Optimize long-context prefill and decode workloads based on real production traffic.
• Tune routing between our infrastructure and external providers by cost, capacity, and performance.
• Work with serving engines such as vLLM, SGLang, and TensorRT-LLM, and go below the framework when needed.
• Build profiling and measurement systems to show where time, memory, and compute are spent.

Skills

ML systems

Inference infra

Performance engineering

Python

C++

Rust

GPU performance

CUDA

NCCL

Quantization

Tools

vLLM

SGLang

TensorRT-LLM

Job description

Adaption is seeking an experienced engineer to own the cost and performance of our inference stack in a production environment. You will influence batching, caching, quantization, and kernel-level optimization while coordinating with the serving fleet to meet throughput and latency targets.

You will optimize prefill/decode workloads, route performance between internal and external providers, and build profiling tools.