Senior GPU Inference Engineer - Ultra-Fast AI Serving

Cerebras; Sunnyvale (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Cerebras Systems is seeking a Software Engineer to productionize and optimize the GPU serving stack, spanning custom inference APIs, the vLLM runtime, ROCm, and rack-scale AMD GPU infrastructure.

You will write production code, establish deployment and health-checking practices, and drive improvements in time to first token, throughput, tail latency, and capacity efficiency across application, runtime, and hardware layers.

Qualifications

• 5+ years of software engineering experience in production systems.
• Experience building/optimizing production inference systems for GPU workloads.
• Strong programming in C++ and Python with multithreading and memory management.
• Hands-on experience with vLLM, ROCm, or TensorRT-LLM or equivalents.
• Knowledge of GPU execution, asynchronous execution, and profiling methods.

Responsibilities

• Productionize GPU inference stack across API services, vLLM, PyTorch, ROCm, and hardware layers.
• Define deployment, upgrades, health-checking, and capacity-management for the AMD GPU fleet.
• Improve reliability with fault isolation, automated recovery, and incident response.
• Profile and optimize time to first token, throughput, and tail latency.
• Tune scheduling, batching, KV-cache management, and distributed communication.

Skills

C++

Python

Multithreading

Performance

Linux

Education

Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, or related discipline

Tools

vLLM

PyTorch

ROCm

TensorRT-LLM

Job description

Cerebras Systems is seeking a Software Engineer to productionize and optimize the GPU serving stack, spanning custom inference APIs, the vLLM runtime, ROCm, and rack-scale AMD GPU infrastructure.

You will write production code, establish deployment and health-checking practices, and drive improvements in time to first token, throughput, tail latency, and capacity efficiency across application, runtime, and hardware layers.