Lead Inference Systems Engineer - AI Serving

adaption

United States

Hybrid

USD 180,000 - 280,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Flexible work
Travel stipend
Lunch stipend
Medical benefits

Job summary

adaption seeks an experienced ML systems engineer to own the cost and performance of our inference stack as workloads, traffic, and hardware evolve. You will work closely with engineers operating the serving fleet, mastering caching, batching, quantization, decoding, and kernel-level optimization to drive efficiency.

Success means higher throughput and lower tail latency without compromising model quality. You’ll shape routing with external providers and build profiling systems to illuminate

Qualifications

  • 5+ years in ML systems, inference infra, or performance engineering.
  • Deep understanding of model serving: prefill, decode, batching, memory bandwidth.
  • Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
  • Strong Python and systems languages: C++, Rust, or similar.
  • Experience with GPU performance: CUDA, NCCL, mixed precision, memory layouts, kernels, quantization.

Responsibilities

  • Improve throughput, cost, and tail latency via KV-cache management, batching, speculative decoding, and quantization.
  • Optimize long-context prefill and decode workloads based on production traffic.
  • Tune routing between infrastructure and external providers for cost, capacity, and performance.
  • Work with serving engines (vLLM, SGLang, TensorRT-LLM) and optimize below framework when needed.
  • Build profiling and measurement systems to show time, memory, and compute usage.

Skills

ML systems
Python
C++
Rust
Performance engineering
GPU performance

Tools

vLLM
SGLang
TensorRT-LLM

Job description

adaption seeks an experienced ML systems engineer to own the cost and performance of our inference stack as workloads, traffic, and hardware evolve. You will work closely with engineers operating the serving fleet, mastering caching, batching, quantization, decoding, and kernel-level optimization to drive efficiency.

Success means higher throughput and lower tail latency without compromising model quality. You’ll shape routing with external providers and build profiling systems to illuminate

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Inference Engineer: Maximize Throughput, Cut Latency & Cost
Inference Engineer: Maximize Throughput, Cut Latency & Cost

adaption • San Francisco (CA)

On-site
USD 150,000 - 230,000
Flexible work: In-person Bay Area coll
Adaption Passport: travel stipend
Lunch Stipend: weekly meal allowance
+1
Lead Inference Engineer — AI Serving & GPU Optimization
Lead Inference Engineer — AI Serving & GPU Optimization

Aleroy • United States

Remote
USD 140,000 - 190,000
Lead Inference Engineer (Fully Remote)
Lead Inference Engineer (Fully Remote)

Aleroy • United States

Remote
USD 140,000 - 190,000
Inference Optimization Engineer: Fast, Cost-Effective ML
Inference Optimization Engineer: Fast, Cost-Effective ML

Build AI • San Francisco (CA)

On-site
USD 150,000 - 210,000
Competitive pay
Medical, dental, and vision packages
Housing subsidy $2k/month near SF offi
+6
Senior ML Inference Systems Engineer
Senior ML Inference Systems Engineer

castaigroupinc • United States

Remote
USD 180,000 - 280,000
Remote-first work
equity options
a learning budget
+3
Founding Inference Systems Engineer
Founding Inference Systems Engineer

Touchdown Labs • United States

Remote
USD 140,000 - 210,000
Staff Inference Engineer — GPU-Optimized ML Serving
Staff Inference Engineer — GPU-Optimized ML Serving

Uniphore • Palo Alto (CA)

On-site
USD 190,000 - 240,000
Annual incentive
Pre-IPO stock options
Medical, dental, vision
+2
Inference Systems Engineer — High-Performance AI Serving
Inference Systems Engineer — High-Performance AI Serving

River AI • Palo Alto (CA)

On-site
USD 200,000 - 420,000
Health insurance
Dental insurance
Vision insurance
+3
Performance Engineer, Inference Systems
Performance Engineer, Inference Systems

Anthropic Limited • San Francisco (CA)

Hybrid
USD 350,000 - 850,000
Performance Engineer, Inference Systems
Performance Engineer, Inference Systems

Anthropic Limited • Seattle (WA)

Hybrid
USD 350,000 - 850,000