Senior Distributed Systems Engineer — Kubernetes & AI Infra

Engg; Toronto; On-site; CAD 140,000 - 210,000; Full time

Job summary

Cerebras Systems is seeking a seasoned cluster engineer to join the team responsible for turning thousands of wafers, servers, and switches into a reliable, observable cloud. The role focuses on automating bare-metal provisioning, upgrades, and security patches, with a strong emphasis on CRDs, controllers, and RBAC in a Go/Python/Kubernetes stack.

You will design and operate large inference workloads, build telemetry pipelines with Prometheus and Grafana, and contribute to resilient,

Qualifications

• 5+ years building and operating production distributed systems or infrastructure software.
• Production-quality Go and Python.
• Real Kubernetes depth: controllers, CRDs, RBAC, and reconciliation semantics.
• Strong debugging skills across distributed systems, Linux, and networking.
• Prometheus and Grafana experience; metrics and exporter design.

Responsibilities

• Declarative, CRD-driven automation of bare-metal networking, OS, and application software across clusters of Cerebras systems, servers, and switches, built to reconcile thousands of nodes.
• Push-button cluster install, upgrade, and security patching with real downtime budgets, gated by canaries.
• Kubernetes operators that schedule large inference workloads: resource locks, priority queues, topology placement, health-aware placement.
• gRPC control-plane services, authorization, admission webhooks, and quota policy for a multi-tenant fleet.
• Metrics and log pipelines with exporters for wafer-scale systems, servers, and network fabric on Prometheus and Grafana, with SLOs and alerting.
• Failure detection, HA control planes, and automated recovery, plus the CLIs, APIs, and MCP gateway that expose the fleet to users, operators, and AI agents.

Skills

distributed systems

Go & Python

Kubernetes depth

debugging distributed systems

Prometheus & Grafana

self-driving capability

AI tooling adoption

infrastructure automation

Job description

Cerebras Systems is seeking a seasoned cluster engineer to join the team responsible for turning thousands of wafers, servers, and switches into a reliable, observable cloud. The role focuses on automating bare-metal provisioning, upgrades, and security patches, with a strong emphasis on CRDs, controllers, and RBAC in a Go/Python/Kubernetes stack.

You will design and operate large inference workloads, build telemetry pipelines with Prometheus and Grafana, and contribute to resilient,