Inference AI Engineer — Scalable GPU ML Services

CoreWeave; Sunnyvale (CA); On-site; USD 92,000 - 135,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401(k) match

Equity awards

ESPP

Flexible PTO

Parental leave

Tuition reimbursement

Catered lunch

Job summary

CoreWeave is the essential cloud for AI, delivering a GPU-focused platform to accelerate breakthroughs. As an IC1, you’ll ship well-scoped features in Python/Go/C++, learn our practices, and grow quickly with mentorship while contributing to model-serving services.

You’ll write tests, reviews, runbooks, and dashboards, participate in incident response, and contribute to performance experiments. The base salary range is $92,000 to $135,000, with a discretionary bonus, equity awards and a full

Qualifications

• BS/MS in CS, EE, or related field, or equivalent practical experience.
• Foundations in data structures, algorithms, and networked services.
• Experience with Python or Go (C++ a plus) and Linux fundamentals; Git/CI basics.
• Exposure to containers and Kubernetes (coursework or projects welcome).
• Curiosity about GPU inference concepts (micro-batching, KV cache, streaming).

Responsibilities

• Implement well-scoped features and fixes in Python/Go/C++ for model-serving services (e.g., Triton, vLLM, TensorRT-LLM, Ray Serve).
• Write tests, code comments, and short design docs; participate in code reviews.
• Add basic metrics and dashboards; assist with alarms and runbooks.
• Follow on-call runbooks and learn incident response in a guided rotation.
• Contribute to performance experiments (e.g., request batching, concurrency, caching) with guidance.

Skills

Python

Go

C++

Data structures

Algorithms

Linux basics

Git basics

Kubernetes

Networking basics

Education

BS in CS/EE

MS in CS/EE

Equivalent practical experience

Tools

Triton

vLLM

TensorRT-LLM

Ray Serve

Docker

Grafana

Prometheus

OpenTelemetry

Job description

CoreWeave is the essential cloud for AI, delivering a GPU-focused platform to accelerate breakthroughs. As an IC1, you’ll ship well-scoped features in Python/Go/C++, learn our practices, and grow quickly with mentorship while contributing to model-serving services.

You’ll write tests, reviews, runbooks, and dashboards, participate in incident response, and contribute to performance experiments. The base salary range is $92,000 to $135,000, with a discretionary bonus, equity awards and a full