Senior AI Inference Platform Engineer

Fdetimes; Sunnyvale (CA); On-site; USD 152,000 - 204,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

Company-paid Life Insurance

401(k) with generous employer match

Paid Parental Leave

Flexible PTO

Job summary

CoreWeave is the Essential Cloud for AI, seeking senior engineers to lead architecture across multiple services and improve latency, throughput, and reliability on our Kubernetes-native inference platform.

You will mentor IC1/IC2 engineers, define SLIs/SLOs, and drive optimizations such as batching, caching, and memory bandwidth utilization while collaborating with product, orchestration, and hardware teams.

Qualifications

• 5-8 years of industry experience building distributed systems or cloud services.
• Strong coding in Python or Go; C++ a plus.
• Experience with inference internals: batching, caching, BF16/FP8.
• Hands-on experience with Kubernetes at production scale, CI/CD, observability.

Responsibilities

• Lead design reviews and drive architecture within the team; decompose multi-service work into milestones.
• Define and own SLIs/SLOs; ensure post-incident actions land and reliability improves release-over-release.
• Implement advanced optimizations (e.g., micro-batch schedulers, speculative decoding) and quantify impact.
• Mentorl IC1/IC2 engineers; review cross-team designs and elevate coding/testing standards.
• Own an area spanning multiple services and teams (e.g., request routing & adaptive scheduling, cost-per-token analytics).

Skills

Python

Go

C++

Networking

CI/CD

Distributed systems

Tools

Kubernetes

Prometheus

Grafana

OpenTelemetry

CUDA

Job description

CoreWeave is the Essential Cloud for AI, seeking senior engineers to lead architecture across multiple services and improve latency, throughput, and reliability on our Kubernetes-native inference platform.

You will mentor IC1/IC2 engineers, define SLIs/SLOs, and drive optimizations such as batching, caching, and memory bandwidth utilization while collaborating with product, orchestration, and hardware teams.