Staff Software Engineer, GPU Inference

Cerebras; Sunnyvale (CA); On-site; USD 180,000 - 280,000; Full time

Job summary

Cerebras Systems is hiring a Software Engineer to productionize and optimize the GPU serving stack across custom inference APIs, vLLM, and ROCm-based infrastructure. This hands-on role emphasizes performance, observability, and reliability in a disaggregated AI inference environment.

You will work across application, runtime, distributed systems, and hardware layers to boost time-to-first-token, throughput, tail latency, and capacity efficiency on the Cerebras Wafer-Scale Engine.

Qualifications

• Minimum 8+ years of software engineering experience in complex production systems.
• Experience building/operating/optimizing production inference systems for large language or GPU workloads.
• Strong programming in C++ and Python with attention to concurrency and performance.

Responsibilities

• Produce and optimize the GPU inference stack from API services to model-serving workers.
• Ensure GPU fleet operational readiness with deployment, upgrades and rollback strategies.
• Define SLIs/SLAs and improve fault isolation, automated recovery, and incident response.

Skills

C++

Python

Multithreading

Distributed systems

GPU performance

Linux

Kubernetes

Benchmarks

Communication

Leadership

Education

Bachelor’s degree in CS/CE/EE

Tools

vLLM

TensorRT-LLM

Triton Inference Server

HIP/ROCm

Docker

Job description

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services.

This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation.

Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.

About The Role

Cerebras is building a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine.

We are hiring a Software Engineer to productionize and optimize our GPU serving stack, working across our custom inference APIs, the vLLM serving runtime, the AMD ROCm software stack, and rack-scale AMD GPU infrastructure, to make this new serving path reliable, numerically correct, observable, and exceptionally performant.

You will write production code, establish operational practices for a new accelerator fleet, and drive improvements in time to first token, throughput, tail latency, and capacity efficiency. This is a hands‑on role requiring deep debugging and optimization across application, runtime, distributed systems, and hardware layers.

Responsibilities

• Productionize the GPU inference stack. Design, build, deploy, and maintain the complete GPU prefill path, spanning API services, model-serving workers, vLLM, PyTorch, ROCm, GPU nodes, networking and rack‑scale infrastructure.
• Own GPU operational readiness. Establish deployment, upgrade, rollback, health‑checking, capacity‑management, and failure‑recovery practices for the AMD GPU fleet. Build automation that makes driver, firmware, runtime, model and container compatibility explicit and reproducible.
• Drive reliability in production. Define service‑level indicators and objectives for GPU‑backed inference. Improve fault isolation, graceful degradation, automated recovery, incident response, and post‑incident remediation across the serving stack.
• Improve inference performance. Profile and optimize time to first token, request throughput, tokens per second per GPU, tail latency, GPU utilization, memory efficiency, and rack‑level capacity under representative production workloads.
• Optimize model‑serving behavior. Tune and improve scheduling, continuous batching, prefix caching, KV‑cache management, tensor and expert parallelism, request admission, quantization, graph execution, and distributed communication.
• Debug across system layers. Diagnose complex failures and performance regressions across application code, vLLM, PyTorch, ROCm/HIP, collective communication libraries, kernels, drivers, firmware, networking and hardware.
• Ensure numerical correctness. Build validation and regression infrastructure for model quality, numerical accuracy, precision changes, quantization, determinism and compatibility across software and hardware releases.
• Build performance and correctness infrastructure. Develop representative benchmarks, workload replay tools, profiling automation, release qualification, dashboards, and regression gates. Turn one‑off investigations into repeatable engineering systems.

Minimum Qualifications

• 8+ years of software engineering experience, including substantial individual‑contributor ownership of complex production systems.
• Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads.
• Strong programming ability in C++ and Python, including experience with multithreading, concurrency, memory management and performance‑sensitive software.
• Hands‑on experience with a high‑performance model‑serving framework such as vLLM, SGLang, TensorRT‑LLM, Triton Inference Server or an equivalent internally developed system.
• Strong understanding of GPU execution and performance, including asynchronous execution, memory movement, synchronization, kernel launches, communication overhead and profiling methodology.
• Experience debugging distributed systems across multiple layers rather than treating the serving framework or accelerator runtime as a black box.
• Experience with Linux, containers, Kubernetes or comparable orchestration systems, observability, CI/CD, and operating latency‑sensitive services in production.
• Ability to design rigorous benchmarks, interpret noisy performance results, identify bottlenecks and translate findings into production improvements.
• Strong communication and technical leadership skills, with a demonstrated ability to drive ambiguous cross‑functional projects to completion.
• Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.

Preferred Qualifications

• Experience with AMD Instinct accelerators and the ROCm ecosystem, including HIP, RCCL, rocprofiler, AMD SMI, AITER, hipBLASLt, Composable Kernel, or related libraries and tools.
• Deep CUDA experience that demonstrates an ability to transfer GPU systems knowledge across accelerator platforms.
• Experience modifying or contributing to vLLM, SGLang, PyTorch, Triton, TensorRT‑LLM, or another open‑source ML systems project.
• Experience optimizing pre