Inference Performance Engineer - Benchmark & Optimize

CoreWeave; Sunnyvale (CA); On-site; USD 188,000 - 275,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401(k) with employer match

Paid parental leave

Flexible PTO

Equity options

Job summary

CoreWeave, The Essential Cloud for AI, is seeking an Applied AI Engineer to improve the real-world performance of our inference platform. You will build benchmarks, profile model-serving pipelines, identify bottlenecks, and drive targeted optimizations across frameworks, runtimes, and hardware.

This role emphasizes empirical evaluation, quantization, caching, and deployment strategies, with opportunities to collaborate with inference engineers to productionize changes and deliver measurable

Qualifications

• 4+ years of ML, systems, performance engineering or adjacent applied engineering work.
• Strong programming skills in Python and production engineering familiarity.
• Experience running empirical evaluations, benchmarks, or experiments and translating results into engineering decisions.
• Familiarity with LLM inference systems and tools such as vLLM, SGLang, TensorRT-LLM.

Responsibilities

• Build and maintain benchmarking workflows measuring latency, throughput, quality regressions, and cost across models and serving configurations.
• Benchmark the inference stack against realistic workloads to find gaps and improvements.
• Profile model-serving behavior across frameworks, runtimes, and hardware to identify bottlenecks in prefill, decode, KV cache, batching, and quantization.
• Drive targeted optimization efforts for customer workloads, including tuning serving configs and evaluating runtime features.

Skills

Python

ML performance

Benchmarks

Clear communication

Tools

vLLM

SGLang

TensorRT-LLM

Nsight Systems

PyTorch profilers

Job description

CoreWeave, The Essential Cloud for AI, is seeking an Applied AI Engineer to improve the real-world performance of our inference platform. You will build benchmarks, profile model-serving pipelines, identify bottlenecks, and drive targeted optimizations across frameworks, runtimes, and hardware.

This role emphasizes empirical evaluation, quantization, caching, and deployment strategies, with opportunities to collaborate with inference engineers to productionize changes and deliver measurable