Staff AI Inference Engineer – Kubernetes-Scale

CoreWeave; Singapore; On-site; SGD 180,000 - 260,000; Full time

Job summary

CoreWeave is seeking a Staff Software Engineer (IC5) for the Inference team in Singapore. You will lead architecture, optimize latency and throughput, and guide cross‑team initiatives across Kubernetes‑based infrastructure.

You will work hands-on on distributed systems, scheduling, batching, memory optimization, and real‑time inference workloads, shaping the performance and reliability of our AI platform at scale.

Qualifications

• 8–12+ years of experience building and operating large-scale distributed systems or cloud platforms.
• Proven experience leading cross-team technical initiatives impacting multiple services or organizations
• Strong programming skills in Go, Python, or C++
• Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design
• Strong understanding of distributed systems, networking, and performance optimization
• Experience designing and operating low-latency, high-throughput systems with strict P95/P99 latency requirements
• Hands-on experience with inference systems, including batching or micro-batching strategies, caching, and memory optimization
• Familiarity with mixed precision (BF16, FP8) and streaming inference workloads

Responsibilities

• Lead architecture, performance, and reliability across multiple services and teams.
• Lead cross-team design initiatives.
• Optimize inference performance (latency, throughput, GPU utilization).
• Improve system reliability at scale.
• Work deeply in distributed systems and Kubernetes-based infrastructure.
• Focus on scheduling, batching, and memory optimization.

Skills

Distributed systems

Cross-team leadership

Go

Python

C++

Kubernetes

Latency optimization

Performance tuning

Inference systems

Tools

Triton

TorchServe

vLLM

Ray Serve

CUDA

NCCL

RDMA

NUMA

Job description

CoreWeave is seeking a Staff Software Engineer (IC5) for the Inference team in Singapore. You will lead architecture, optimize latency and throughput, and guide cross‑team initiatives across Kubernetes‑based infrastructure.

You will work hands-on on distributed systems, scheduling, batching, memory optimization, and real‑time inference workloads, shaping the performance and reliability of our AI platform at scale.