Staff Software Engineer, AI Inference Systems

Cerebras; Toronto; On-site; CAD 140,000 - 190,000; Full time

Job summary

Cerebras Systems builds the world's largest AI chip and is hiring a Software Engineer to build and evolve the ML API layer for a disaggregated inference system. You will work across inference APIs, model integration, request-routing services, and Cerebras runtimes to deliver a consistent experience across models and accelerator backends.

This hands-on role touches streaming, tool use, structured outputs, multimodal inputs, and production-quality APIs, collaborating with model enablement,

Qualifications

• 5+ years of software engineering in production software or distributed systems.
• Strong Python and Go plus experience with performance-sensitive services in C++, Rust, or a similar language.

Responsibilities

• Build production ML inference APIs for chat, text generation, streaming, model configuration, tool calling, and structured outputs.
• Deliver a unified serving experience across GPU prefill, Cerebras decode, and other backends.
• Enable new models and capabilities by integrating foundation models, tokenizers, prompts, sampling, and multimodal inputs.
• Own API compatibility and evolution with versioning, deprecation, and backward compatibility practices.
• Integrate with model-serving runtimes like vLLM, PyTorch, Hugging Face libraries, ROCm stack, and Cerebras runtime components.
• Support disaggregated inference with routing, state transfer, error handling, retries, and lifecycle management.
• Improve serving performance: streaming, latency, throughput, batching, serialization, and scheduling.
• Ensure correctness through validation systems for tokenization, sampling, logits, and model upgrades.
• Strengthen reliability with logging, tracing, metrics, dashboards, health checks, and diagnostics.
• Develop testing and qualification infrastructure, conformance tests, workload replay, and performance benchmarks.
• Improve developer experience with configuration, SDKs, documentation, examples, and self-service tooling.
• Collaborate across stack with compiler, runtime, kernel, cloud, product, and solutions teams to translate requirements into scalable serving capabilities.

Skills

Python

Go

C++

Rust

APIs design

Linux

Kubernetes

OpenAI APIs

Distributed systems

Performance

Education

BS in CS/CE/EE

Tools

vLLM

PyTorch

Hugging Face

Triton

TensorRT-LLM

Job description

Cerebras Systems builds the world's largest AI chip and is hiring a Software Engineer to build and evolve the ML API layer for a disaggregated inference system. You will work across inference APIs, model integration, request-routing services, and Cerebras runtimes to deliver a consistent experience across models and accelerator backends.

This hands-on role touches streaming, tool use, structured outputs, multimodal inputs, and production-quality APIs, collaborating with model enablement,