Senior Inference Platform Engineer (Kubernetes)

Fdetimes; Sunnyvale (CA); On-site; USD 188,000 - 275,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401(k) plan

Employer match

Paid parental leave

Flexible PTO

Tuition reimbursement

ESPP

Health Savings Account

Job summary

CoreWeave is seeking a Staff Software Engineer (IC5) to lead architecture and performance on the Inference platform. You will drive cross-team initiatives, optimize latency and throughput, and improve system reliability at scale within a Kubernetes-based infrastructure.

You will work with Go, Python, and C++ as you shape distributed systems, scheduling, batching, memory management, and GPU utilization, while influencing engineering direction and mentoring other engineers in a fast-paced cloud

Qualifications

• 8–12+ years of experience building and operating large-scale distributed systems or cloud platforms.
• Proven experience leading cross-team technical initiatives impacting multiple services or organizations.
• Strong programming skills in Go, Python, or C++.
• Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design.
• Strong understanding of distributed systems, networking, and performance optimization.
• Experience designing and operating low-latency, high-throughput systems with strict P95/P99 latency requirements.
• Hands‑on experience with inference systems, including batching or micro‑batching strategies, caching, and memory optimization.
• Experience improving system performance using metrics‑driven approaches (e.g., latency, throughput, utilization).
• Familiarity with mixed precision (BF16, FP8) and streaming inference workloads.

Responsibilities

• Lead architecture and reliability across multiple services and teams.
• Optimize inference performance (latency, throughput, GPU utilization) and system reliability at scale.
• Drive cross-team design initiatives in distributed systems and Kubernetes-based infrastructure.
• Focus on scheduling, batching, and memory optimization for real-time inference workloads.
• Mentor engineers and influence engineering direction across the organization.

Skills

Go

Python

C++

Kubernetes

Distributed systems

Latency optimization

Performance tuning

Inference systems

Cross-team leadership

Tools

CUDA

NCCL

RDMA

NUMA

GPU interconnects

Triton

vLLM

TorchServe

Job description

CoreWeave is seeking a Staff Software Engineer (IC5) to lead architecture and performance on the Inference platform. You will drive cross-team initiatives, optimize latency and throughput, and improve system reliability at scale within a Kubernetes-based infrastructure.

You will work with Go, Python, and C++ as you shape distributed systems, scheduling, batching, memory management, and GPU utilization, while influencing engineering direction and mentoring other engineers in a fast-paced cloud