Staff GPU Inference Engineer — Real-Time AI Systems

Cerebras; United States; Remote; USD 150,000 - 230,000; Full time

Job summary

Cerebras Systems is hiring a Software Engineer to productionize and optimize our GPU serving stack, spanning API services, model-serving workers, vLLM, PyTorch, ROCm, and rack-scale AMD GPU infrastructure.

This hands-on role requires deep debugging and optimization across application, runtime, distributed systems, and hardware layers to improve time to first token, throughput, tail latency, and capacity efficiency.

Qualifications

• Experience building production-grade GPU serving systems.
• Strong debugging and optimization across software and hardware.
• Familiarity with model serving and distributed computation.

Responsibilities

• Productionize the GPU inference stack. Design, build, deploy, and maintain the complete GPU prefill path, spanning API services, model-serving workers, vLLM, PyTorch, ROCm, GPU nodes, networking, and rack-scale infrastructure.
• Own GPU operational readiness. Establish deployment, upgrade, rollback, health-checking, capacity-management, and failure-recovery practices for the AMD GPU fleet.
• Drive reliability in production. Define service-level indicators and objectives for GPU-backed inference; improve fault isolation, automated recovery, incident response, and remediation.
• Improve inference performance. Profile and optimize time to first token, throughput, tail latency, and GPU utilization.
• Optimize model-serving behavior. Tune scheduling, batching, KV-cache management, parallelism, and distributed communication.
• Debug across system layers. Diagnose failures across application code, vLLM, PyTorch, ROCm/HIP, and hardware.

Skills

GPU inference optimization

Debugging across system layers

Distributed systems

Python/C++ production code

Tools

PyTorch

vLLM

ROCm/HIP

GPU tooling

Job description

Cerebras Systems is hiring a Software Engineer to productionize and optimize our GPU serving stack, spanning API services, model-serving workers, vLLM, PyTorch, ROCm, and rack-scale AMD GPU infrastructure.

This hands-on role requires deep debugging and optimization across application, runtime, distributed systems, and hardware layers to improve time to first token, throughput, tail latency, and capacity efficiency.