Inference AI/ML Engineer for GPU Model Serving

re-zoo-me; Sunnyvale (CA); On-site; USD 92,000 - 135,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

Life Insurance

ESPP

Tuition Reimbursement

401(k) with employer match

Flexible PTO

Catered lunch

Mental wellness benefits

Job summary

CoreWeave is seeking an IC1 engineer to join the Inference team and ship production features for model serving on our GPU platform. You will implement well-scoped changes, learn our practices, and grow quickly with mentorship from experienced engineers.

You will work on Python/Go/C++ services such as Triton, vLLM, and TensorRT-LLM, write tests and docs, and contribute to metrics, dashboards, and runbooks. This is an opportunity to advance in a fast-paced AI infrastructure company in Sunnyvale,

Qualifications

• BS/MS in CS, EE, or related field, or equivalent practical experience.
• Foundations in data structures, algorithms, and networked services.
• Experience with Python or Go; C++ is a plus; Linux fundamentals; Git/CI basics.
• Exposure to containers and Kubernetes (coursework or projects welcome).
• Curiosity about GPU inference concepts (micro-batching, KV cache, streaming).

Responsibilities

• Implement features and fixes in Python/Go/C++ for model-serving services (e.g., Triton, vLLM, TensorRT-LLM, Ray Serve).
• Write tests, code comments, and short design docs; participate in code reviews.
• Add basic metrics and dashboards; assist with alarms and runbooks.
• Follow on-call runbooks and learn incident response in a guided rotation.
• Contribute to performance experiments (e.g., request batching, concurrency, caching) with guidance.

Skills

Python/Go

C++

Linux

Git/CI

Containers/Kubernetes

GPU inference concepts

Education

BS/MS in CS/EE or related

Tools

Triton

vLLM

TensorRT-LLM

Ray Serve

Grafana/Prometheus/OpenTelemetry

Job description

CoreWeave is seeking an IC1 engineer to join the Inference team and ship production features for model serving on our GPU platform. You will implement well-scoped changes, learn our practices, and grow quickly with mentorship from experienced engineers.

You will work on Python/Go/C++ services such as Triton, vLLM, and TensorRT-LLM, write tests and docs, and contribute to metrics, dashboards, and runbooks. This is an opportunity to advance in a fast-paced AI infrastructure company in Sunnyvale,