Inference Engineer

adaption

United States

Hybrid

USD 180,000 - 280,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Flexible work
Travel stipend
Lunch stipend
Medical benefits

Job summary

adaption seeks an experienced ML systems engineer to own the cost and performance of our inference stack as workloads, traffic, and hardware evolve. You will work closely with engineers operating the serving fleet, mastering caching, batching, quantization, decoding, and kernel-level optimization to drive efficiency.

Success means higher throughput and lower tail latency without compromising model quality. You’ll shape routing with external providers and build profiling systems to illuminate

Qualifications

  • 5+ years in ML systems, inference infra, or performance engineering.
  • Deep understanding of model serving: prefill, decode, batching, memory bandwidth.
  • Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
  • Strong Python and systems languages: C++, Rust, or similar.
  • Experience with GPU performance: CUDA, NCCL, mixed precision, memory layouts, kernels, quantization.

Responsibilities

  • Improve throughput, cost, and tail latency via KV-cache management, batching, speculative decoding, and quantization.
  • Optimize long-context prefill and decode workloads based on production traffic.
  • Tune routing between infrastructure and external providers for cost, capacity, and performance.
  • Work with serving engines (vLLM, SGLang, TensorRT-LLM) and optimize below framework when needed.
  • Build profiling and measurement systems to show time, memory, and compute usage.

Skills

ML systems
Python
C++
Rust
Performance engineering
GPU performance

Tools

vLLM
SGLang
TensorRT-LLM

Job description

The role

You'll own the cost and performance of our inference stack. Your work will determine how efficiently we serve models as workloads, traffic, and hardware change.

You'll work closely with the engineers operating the serving fleet while owning the core performance levers: caching, batching, quantization, decoding, and kernel-level optimization.

Success means improving throughput and latency without compromising reliability or model quality.

Responsibilities
  • Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.
  • Optimize long-context prefill and decode workloads based on real production traffic.
  • Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
  • Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
  • Build profiling and measurement systems that show where time, memory, and compute are being spent.
Qualifications
  • 5+ years in ML systems, inference infrastructure, or performance engineering, with measurable improvements in cost or latency.
  • Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
  • Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
  • Strong Python skills and proficiency in C++, Rust, or another systems language.
  • Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.

Above all, we're looking for great teammates who make work feel lighter and aren't afraid to go out on a limb with bold ideas. You don't need to be perfect, but you do need to be adaptable. We encourage you to apply, even if you don't check every box.

About us

Most AI is frozen in place - it doesn't adapt to the world. We think that's backwards. Our mandate is to build efficient intelligence that evolves in real-time. Our vision is AI systems that are flexible, personalized, and accessible to everyone. We believe efficiency is what makes this possible - it's how we expand access and ensure innovation benefits the many, not the few. We believe in talent density: bringing together the best and most driven individuals to push the boundaries of continual adaptation. We're looking for builders and creative thinkers ready to shape the next era of intelligence.

Benefits
  • Flexible work : In-person collaboration in the Bay Area, a distributed global-first team, and team offsites.
  • Adaption Passport : Annual travel stipend to explore a country you've never visited. We're building intelligence that evolves alongside you, so we encourage you to keep expanding your horizons.
  • Lunch Stipend: Weekly meal allowance for take-out or grocery delivery.
  • Well-Being : Comprehensive medical benefits and generous paid time off.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Inference Engineer
Inference Engineer

adaption • San Francisco (CA)

On-site
USD 150,000 - 230,000
Flexible work: In-person Bay Area coll
Adaption Passport: travel stipend
Lunch Stipend: weekly meal allowance
+1
Platform Engineer: APIs & Security (Multi-Tenant) Hybrid
Platform Engineer: APIs & Security (Multi-Tenant) Hybrid

Adaption Labs, Inc. • San Francisco (CA)

Hybrid
USD 170,000 - 230,000
Flexible work
Travel stipend
Lunch stipend
+2
Low-Latency Inference Infrastructure Engineer
Low-Latency Inference Infrastructure Engineer

Elorian • Palo Alto (CA)

Hybrid
USD 200,000 - 400,000
Health, dental, and vision benefits
Unlimited PTO
Paid parental leave
+1
Staff / Principal Machine Learning Engineer, Serving
Staff / Principal Machine Learning Engineer, Serving

Inworld AI • Mountain View (CA)

On-site
USD 270,000 - 500,000
Relocation assistance
Equity options
Comprehensive benefits package
Inference Engineer - Member of Technical Staff
Inference Engineer - Member of Technical Staff

Designworks Talent • Bellevue (WA)

Hybrid
USD 150,000 - 230,000
Medical, dental, and vision insurance
401(k) with company match
Paid holidays
Senior Software Engineer - Model Performance
Senior Software Engineer - Model Performance

inference.net • San Francisco (CA)

Hybrid
USD 220,000 - 320,000
Equity in a high-growth startup
Comprehensive benefits
Research Scientist
Research Scientist

adaption • New York (NY)

On-site
USD 180,000 - 260,000
Flexible work
Annual travel stipend
Lunch stipend
+1
Research Scientist
Research Scientist

adaption • United States

On-site
USD 190,000 - 240,000
Flexible work arrangements
Travel stipend
Lunch stipend
+1
Senior Software Engineer - Model Performance
Senior Software Engineer - Model Performance

Inference • San Francisco (CA)

Hybrid
USD 220,000 - 320,000
Competitive compensation
Equity in a high-growth startup
Comprehensive benefits
Tech Lead - AI Inference
Tech Lead - AI Inference

wekatest • United States

Remote
USD 180,000 - 240,000
Medical insurance
Dental insurance
Vision insurance
+5