Senior Kubernetes & AI Inference Platform Engineer

CoreWeave; Sunnyvale (CA); On-site; USD 139,000 - 204,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

Company-paid Life Insurance

Equity / ESPP

Tuition Reimbursement

401(k) with generous match

Flexible PTO

Parental Leave

Lunch provided

Job summary

CoreWeave, The Essential Cloud for AI, seeks senior engineers to own design and reliability across a multi-service inference platform. You’ll drive architecture reviews, set SLI/SLO targets, and implement advanced optimizations while mentoring IC1/IC2 engineers.

Roles span IC3–IC4 with opportunities to influence routing, scheduling, and GPU isolation. You’ll collaborate with product, orchestration, and hardware teams to meet strict P99 SLAs at scale, delivering measurable latency and throughput

Qualifications

• BS or MS in CS, EE, or related field.
• 3–8 years of distributed systems or cloud service experience.
• Strong coding in Python or Go (C++ a plus).
• Hands-on Kubernetes at production scale, CI/CD, and observability stacks.

Responsibilities

• Lead design reviews and architecture within the team; decompose multi-service work into milestones.
• Define and own SLIs/SLOs; ensure post-incident actions land and reliability improves release-over-release.
• Implement optimizations (micro-batch schedulers, speculative decoding, KV-cache reuse) and quantify impact.
• Strengthen incident posture: capacity planning, autoscaling policy, graceful degradation, rollback/traffic-shift strategies.
• Mentor IC1/IC2 engineers; review cross-team designs and elevate coding/testing standards.
• For IC4: own an area spanning multiple services and teams (e.g., request routing & adaptive scheduling, cost-per-token analytics, GPU resource isolation).

Skills

Python/Go

Kubernetes

CI/CD

Observability

Batching

Caching

BF16/FP8

Tail latency

Education

Bachelor's or Master’s in CS/EE

Tools

Prometheus

Grafana

OpenTelemetry

Job description

CoreWeave, The Essential Cloud for AI, seeks senior engineers to own design and reliability across a multi-service inference platform. You’ll drive architecture reviews, set SLI/SLO targets, and implement advanced optimizations while mentoring IC1/IC2 engineers.

Roles span IC3–IC4 with opportunities to influence routing, scheduling, and GPU isolation. You’ll collaborate with product, orchestration, and hardware teams to meet strict P99 SLAs at scale, delivering measurable latency and throughput