Principal Engineer, AI Inference Reliability

Cerebras; Raleigh (NC); On-site; USD 170,000 - 250,000; Full time

Job summary

Cerebras Systems is seeking a hands-on Reliability Tech Lead to own the mission of making Cerebras Inference the most reliable AI service. You will establish SLOs and drive reliability across the inference stack, from client SDKs to data-center deployments.

You will define incident-response frameworks, design scalable reliability mechanisms, and collaborate with hundreds of engineers to uphold world-class reliability standards.

Qualifications

• 7+ years of backend, infrastructure, or reliability engineering for large-scale distributed systems.
• Strong programming skills in Python, C++, Go, or Rust.
• Deep experience with reliability: SLO/SLI/SLA design, incident response, postmortem culture.
• Excellent communication and cross-functional leadership skills.
• Bonus: experience building large-scale AI infrastructure systems.

Responsibilities

• Define and drive reliability strategy: establish SLOs and ensure alignment across engineering.
• Design and implement reliability mechanisms: build, evolve systems for fault detection, degradation, failover, throttling, and recovery across multiple regions and data centers.
• Lead large-scale incident management: own postmortems, root-cause analysis, and prevention loops for reliability incidents.
• Architect for reliability and observability: influence system design for redundancy, durability, and debuggability.
• Develop reliability tooling: create internal tools and frameworks for chaos testing, load simulation, and distributed fault injection.
• Collaborate broadly: work across software, infrastructure, and hardware teams to embed reliability across layers of the inference service.
• Monitor and communicate reliability metrics: build dashboards and alerts for service health and actionable insights.
• Mentor and influence: guide engineers and share best practices for designing, testing, and operating reliable large-scale systems.

Skills

Backend engineering

Distributed systems

SLO/SLI design

Incident response

Leadership

Python

C++

Go

Rust

Education

Bachelor's degree in CS

Master's degree in CS

Job description

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services.

This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real‑time iteration and increasing intelligence via additional agentic computation.

Cerebras works with the leading model labs, global enterprises, and cutting‑edge AI‑native startups. OpenAI recently announced a multi‑year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high‑speed inference.

About The Role

We’re looking for a hands‑on Reliability Tech Lead (IC) to own the mission of making Cerebras Inference the most reliable AI service in the world. You will drive reliability strategy and execution across our inference stack, from client SDKs and public‑cloud multi‑region deployments to wafer‑scale systems in specialized data centers.

In this role, you will define SLOs and incident‑response frameworks, design and implement reliability mechanisms at scale, and partner across hundreds of engineers to ensure our service meets world‑class reliability standards.

If you are passionate about building and operating massive‑scale, low‑latency, high‑reliability distributed systems, we want to hear from you.

Responsibilities

• Define and drive reliability strategy: establish SLOs and ensure alignment across engineering.
• Design and implement reliability mechanisms: build and evolve systems for fault detection, graceful degradation, failover, throttling, and recovery across multiple regions and data centers.
• Lead large‑scale incident management: own postmortems, root‑cause analysis, and prevention loops for reliability‑related incidents.
• Architect for reliability and observability: influence system design for redundancy, durability, and debuggability.
• Develop reliability tooling: create internal tools and frameworks for chaos testing, load simulation, and distributed fault injection.
• Collaborate broadly: work across software, infrastructure, and hardware teams to ensure reliability is embedded into every layer of our inference service.
• Monitor and communicate reliability metrics: build dashboards and alerts that measure service health and provide actionable insights.
• Mentor and influence: guide engineers and set best practices for designing, testing, and operating reliable large‑scale systems.

Skills & Qualifications

• Bachelor's or master's degree in computer science or related field.
• 7+ years of experience in backend, infrastructure, or reliability engineering for large‑scale distributed systems.
• Strong programming skills in at least one popular backend programming language such as Python, C++, Go, or Rust.
• Deep and hard‑earned experience of reliability principles: SLO/SLI/SLA design, incident response, and postmortem culture.
• Excellent communication and cross‑functional leadership skills.
• Bonus: prior experience building large‑scale AI infrastructure systems.

Cerebras Systems is committed to creating an equal and diverse environment and is proud to be an equal opportunity employer. We celebrate different backgrounds, perspectives, and skills. We believe inclusive teams build better products and companies. We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around us.

This website or its third‑party tools process personal data. For more details, click here to review our CCPA disclosure notice.