Applied AI Inference Performance Engineer

TheDataJob; San Francisco (CA); On-site; USD 188,000 - 275,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

ESPP – Employee Stock Purchase Program

Tuition Reimbursement

Paid Parental Leave

Flexible PTO

Casual work environment

Job summary

CoreWeave, The Essential Cloud for AI, seeks an Applied AI Engineer to measure and improve the real-world performance of its inference platform. You will build benchmarks, profile model-serving behavior, and drive optimizations across platforms, runtimes, and hardware toward scalable, cost-efficient AI workloads.

The role emphasizes applied performance work with collaboration across teams. Expect to contribute through experiments, validation, and reproducible workflows.

Qualifications

• 4+ years of experience in machine learning, systems, performance engineering, or adjacent applied engineering work.

Responsibilities

• Build and maintain benchmarking workflows measuring latency, throughput, and cost.
• Benchmark inference stack against real customer workloads and baselines.
• Profile model-serving behavior across frameworks, runtimes, and hardware to find bottlenecks.
• Drive targeted optimization for customer workloads and validate changes with traces and benchmarks.
• Design experiments on model-serving techniques like quantization and caching strategies.

Skills

Python programming

Performance benchmarking

Empirical evaluation

Model serving experience

LLM inference systems

Cross-functional collaboration

Tools

Nsight Systems

PyTorch profilers

TensorRT-LLM

vLLM

SGLang

Job description

CoreWeave, The Essential Cloud for AI, seeks an Applied AI Engineer to measure and improve the real-world performance of its inference platform. You will build benchmarks, profile model-serving behavior, and drive optimizations across platforms, runtimes, and hardware toward scalable, cost-efficient AI workloads.

The role emphasizes applied performance work with collaboration across teams. Expect to contribute through experiments, validation, and reproducible workflows.