Senior GPU Kernel Engineer: Inference Performance

Neura Market; Sunnyvale, Northern (CA, KY); Hybrid; USD 182,000 - 242,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

Company-paid Life Insurance

Tuition Reimbursement

Employee Stock Purchase Program (ESPP)

Flexible PTO

Job summary

CoreWeave is seeking a Senior Engineer for its Benchmarking & Performance team to own kernel-level optimization for LLM inference and end-to-end model serving, focusing on CUDA kernels and throughput/latency improvements.

You will lead kernel design reviews, mentor engineers, and drive reproducible benchmarking across vLLM, TensorRT-LLM, and related stacks while partnering with product, orchestration, and hardware teams.

Qualifications

• 5+ years of experience building high-performance computing, GPU/accelerator software, or performance-critical systems.
• Hands-on CUDA experience is required—you have written and optimized custom kernels and are fluent with the CUDA programming and memory model.
• Deep understanding of GPU architecture and performance: tensor cores, warp/occupancy tuning, the memory hierarchy and bandwidth, NVLink/PCIe, and profiling with Nsight Compute/Systems.
• Strong coding in C++ and Python; comfortable reading and writing low-level, performance-sensitive code.
• Familiarity with model-serving stacks (vLLM, TensorRT-LLM, llm-d, SGLang) and the kernels that dominate their inference cost.
• Strong communicator comfortable collaborating with cross-functional teams and external partners.

Responsibilities

• Author, profile, and optimize CUDA kernels—GEMMs, attention, MoE routing, quantization, KV-cache, and fused epilogues—on the critical path of LLM inference.
• Optimize for the hardware: exploit tensor cores and tune occupancy, memory coalescing, shared-memory/register usage, and overlap of compute with data movement.
• Use kernel-authoring DSLs and compilers to prototype and ship kernels quickly without sacrificing performance.
• Benchmark rigorously: build reproducible microbenchmarks and roofline analyses, and validate that kernel-level wins translate to end-to-end latency/throughput gains across model-serving stacks (vLLM, TensorRT-LLM, llm-d, SGLang).
• Implement and maintain benchmarking workflows for end-to-end MLPerf Inference (and Training) runs, including workload setup, cluster configuration, runbooks, and result validation.
• Lead design reviews and drive architecture within the team; decompose multi-service work into clear milestones.
• Mentor junior engineers; review cross-team designs and elevate coding/testing standards.
• Help ensure reproducible, well-documented benchmarking and kernel-optimization processes.

Skills

CUDA kernels

C++

Python

GPU architecture

Performance optimization

Mentoring

Cross-functional collaboration

Tools

Nsight Compute

TensorRT

Job description

CoreWeave is seeking a Senior Engineer for its Benchmarking & Performance team to own kernel-level optimization for LLM inference and end-to-end model serving, focusing on CUDA kernels and throughput/latency improvements.

You will lead kernel design reviews, mentor engineers, and drive reproducible benchmarking across vLLM, TensorRT-LLM, and related stacks while partnering with product, orchestration, and hardware teams.