Senior Software Engineer - AI Inference Platform

CoreWeave; Sunnyvale (CA); On-site; USD 152,000 - 204,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401(k) with employer match

Flexible PTO

Catered lunch

Office snacks

Tuition Reimbursement

ESPP participation

Mental Wellness benefits

Parental leave

Job summary

CoreWeave in California is seeking a Senior Software Engineer to own designs and drive reliability across multi-service inference platforms. You will partner with product, orchestration, and hardware teams to evolve our Kubernetes-native systems and meet strict P99 SLAs at scale.

You will design reviews, own SLIs/SLOs, and implement optimizations such as micro-batch schedulers and caching strategies, while mentoring engineers and improving end-to-end performance of AI workloads.

Qualifications

• 5–8 years of experience building distributed systems or cloud services.
• Strong coding in Python or Go (C++ a plus) and deep knowledge of networked systems.
• Hands-on experience with Kubernetes at production scale, CI/CD, and observability stacks.
• Ability to optimize end-to-end ML system performance and reduce latency.
• Proven track record improving tail latency and reliability through metrics-driven work.

Responsibilities

• Lead design reviews and drive architecture within the team; decompose multi-service work into milestones.
• Define and own SLIs/SLOs; ensure post-incident actions land and reliability improves release-over-release.
• Implement optimizations (micro-batch schedulers, caching, KV-cache reuse) and quantify impact.
• Strengthen incident posture: capacity planning, autoscaling policy, graceful degradation, rollback/traffic-shift strategies.
• Mentor IC1/IC2 engineers; review cross-team designs and elevate coding/testing standards.
• Own an area spanning multiple services and teams (e.g., routing, scheduling, per-token analytics, GPU isolation).

Skills

Python

Go

C++

Kubernetes

CI/CD

Observability

Performance

CUDA

Networking

Tools

Kubernetes

Prometheus

Grafana

OpenTelemetry

CUDA

Job description

CoreWeave in California is seeking a Senior Software Engineer to own designs and drive reliability across multi-service inference platforms. You will partner with product, orchestration, and hardware teams to evolve our Kubernetes-native systems and meet strict P99 SLAs at scale.

You will design reviews, own SLIs/SLOs, and implement optimizations such as micro-batch schedulers and caching strategies, while mentoring engineers and improving end-to-end performance of AI workloads.