Applied AI Inference Performance Engineer

CoreWeave; San Francisco (CA); On-site; USD 188,000 - 275,000; Full time

Benefits offered by this job

Medical benefits

401(k) with match

Flexible PTO

Catered lunch

Tuition Reimbursement

Employee Stock Purchase Program (ESPP)

Parental Leave

Family-forming support

Job summary

CoreWeave, a leading AI cloud provider, seeks an Applied AI Engineer to improve real-world performance of the inference platform. You will benchmark, profile, and optimize across frameworks, runtimes, and GPUs, delivering measurable improvements for customer workloads.

The role focuses on applied performance work, including quantization, decoding, and caching strategies, with close collaboration with the Inference Team. Strong Python skills and 4+ years’ experience are required.

Qualifications

• 4+ years of experience in ML, systems, performance engineering, or related applied engineering work.
• Strong Python programming and production engineering experience.
• Experience running empirical evaluations, benchmarks, or experiments and translating results into concrete engineering decisions.
• Familiarity with LLM inference systems and tools such as vLLM, SGLang, TensorRT-LLM, or similar stacks.
• Understanding practical tradeoffs in latency, throughput, batching, GPU utilization, quantization, and quality regression analysis.
• Ability to work across model, systems, and product boundaries with customer outcomes in mind.
• Strong written communication and reproducibility skills.

Responsibilities

• Build and maintain benchmarking workflows measuring latency, throughput, quality regressions, and cost across models and serving configurations.
• Benchmark inference stack against realistic customer workloads and external baselines to identify gaps.
• Profile model-serving behavior across frameworks, runtimes, and hardware to find bottlenecks.
• Drive targeted optimization for specific workloads, tuning configurations and validating changes with traces and benchmarks.
• Design experiments on quantization, speculative decoding, caching, routing, and other inference optimizations with quality tradeoffs.
• Collaborate with inference platform engineers to productionize improvements and establish repeatable performance testing workflows.
• Produce technical writeups and recommendations for model configurations, runtimes, hardware allocations, and deployment strategies.
• Contribute applied research to support inference quality, optimization, and product performance goals.

Skills

ML/Systems

Python

Benchmarking

LLM inference

Performance analysis

Cross-functional collaboration

Tools

Nsight Systems

PyTorch profilers

Job description

CoreWeave, a leading AI cloud provider, seeks an Applied AI Engineer to improve real-world performance of the inference platform. You will benchmark, profile, and optimize across frameworks, runtimes, and GPUs, delivering measurable improvements for customer workloads.

The role focuses on applied performance work, including quantization, decoding, and caching strategies, with close collaboration with the Inference Team. Strong Python skills and 4+ years’ experience are required.