Staff GPU Inference Engineer - Real-Time AI at Scale

Cerebras Systems; Toronto; On-site; CAD 180,000 - 240,000; Full time

Job summary

Cerebras Systems is hiring a Software Engineer to productionize and optimize our GPU serving stack across vLLM, PyTorch, ROCm, and rack-scale GPU infrastructure. You will write production-grade code, improve time to first token, throughput, and capacity efficiency, and drive reliability in a hands-on role spanning application, runtime, distributed systems, and hardware layers.

The role emphasizes deep debugging, numerical correctness, and automated workflows to make the serving path robust and

Qualifications

• 8+ years of software engineering experience in production systems.
• Experience with large language models or GPU workloads.
• Strong programming in C++ and Python with concurrency.
• Hands-on with a high-performance model-serving framework.
• Understanding of GPU execution and profiling.
• Experience debugging distributed systems.
• Linux, containers, Kubernetes, observability, CI/CD.
• Ability to design benchmarks and interpret results.

Responsibilities

• Productionize GPU inference stack end-to-end.
• Own GPU operational readiness and automation.
• Improve production reliability and incident response.
• Profile and optimize inference time, throughput, tail latency.
• Debug across application to hardware layers.
• Ensure numerical correctness and testing.
• Build benchmarks and regression gates.

Skills

C++

Python

Multithreading

Performance optimization

Distributed systems

Linux

Education

Bachelor’s degree in CS/EE

Tools

vLLM

PyTorch

ROCm

TensorRT-LLM

Job description

Cerebras Systems is hiring a Software Engineer to productionize and optimize our GPU serving stack across vLLM, PyTorch, ROCm, and rack-scale GPU infrastructure. You will write production-grade code, improve time to first token, throughput, and capacity efficiency, and drive reliability in a hands-on role spanning application, runtime, distributed systems, and hardware layers.

The role emphasizes deep debugging, numerical correctness, and automated workflows to make the serving path robust and