Senior GPU Kernel Engineer - High-Performance Inference

CoreWeave; United States; On-site; USD 182,000 - 242,000; Full time

Benefits offered by this job

Medical Insurance

401(k) Matching

Paid Parental Leave

Flexible PTO

Equity Awards

Tuition Reimbursement

Job summary

CoreWeave is the top-rated AI-cloud for high-performance GPU infrastructure. We’re seeking a Senior Engineer for Benchmarking & Performance to own kernel design and optimization on large-scale model serving, squeezing throughput and reducing latency across the inference stack.

You will write, profile, and tune CUDA kernels (GEMMs, attention, KV-cache) and lead end-to-end benchmarking such as MLPerf, partnering with product, orchestration, and hardware teams to meet strict P99 SLAs at scale.

Qualifications

• 5+ years of experience building high-performance computing, GPU/accelerator software, or performance-critical systems.
• Hands-on CUDA experience with custom kernel development and memory model proficiency.
• Strong coding in C++ and Python; reading/writing low-level, performance-sensitive code.
• Familiarity with model-serving stacks (vLLM, TensorRT-LLM, llm-d, SGLang) and the kernels that dominate their inference cost.
• Strong communicator capable of collaborating with cross-functional teams and external partners.

Responsibilities

• Author, profile, and optimize CUDA kernels on the critical path of LLM inference.
• Tune for hardware performance: tensor cores, occupancy, memory hierarchy, data movement overlap.
• Benchmark rigor: build reproducible microbenchmarks and end-to-end latency/throughput validation.
• Lead design reviews and drive architecture; decompose work into milestones.
• Mentor junior engineers and elevate coding/testing standards.

Skills

CUDA kernels

GPU architecture

C++

Python

Nsight Compute

Model-serving stacks

Kubernetes

MLPerf

OSS contributions

KNYFE / DSLs

Tools

Nsight Compute/Systems

TensorRT

Kubernetes (prod)

Job description

CoreWeave is the top-rated AI-cloud for high-performance GPU infrastructure. We’re seeking a Senior Engineer for Benchmarking & Performance to own kernel design and optimization on large-scale model serving, squeezing throughput and reducing latency across the inference stack.

You will write, profile, and tune CUDA kernels (GEMMs, attention, KV-cache) and lead end-to-end benchmarking such as MLPerf, partnering with product, orchestration, and hardware teams to meet strict P99 SLAs at scale.