Engineering Manager, Inference Router: Global Low-Latency

Cerebras

Toronto

On-site

CAD 180,000 - 260,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Cerebras Systems is seeking an Engineering Manager, Inference Router to lead the team that directs where every inference request runs across Cerebras data centers and partners. You will own global routing, cache-aware scheduling, and autoscaling, partnering with ML and cloud teams to optimize latency, throughput, and reliability.

You will build and grow a distributed engineering team, drive architectural decisions, and ensure high-availability, workload resilience, and efficient use of

Qualifications

  • 8+ years in software eng, distributed systems or infra.
  • 2+ years in engineering management.
  • Strong coding in C++ and/or Go for performance-critical systems.
  • Deep experience operating production services on Kubernetes.
  • Proven track record in routing, load balancing, and traffic management.

Responsibilities

  • Own architecture and evolution of the inference routing layer across data centers and third-party integrations.
  • Lead design of multi-region routing, cache-aware routing, and scheduling under bursty load.
  • Drive real-time routing decisions balancing latency and throughput.
  • Build and mentor a team of distributed systems engineers.
  • Collaborate with inference runtime, cloud, hardware, and ML teams.

Skills

8+ years experience
Engineering management
C++/Go programming
Kubernetes
Distributed systems
Routing/traffic management
Cloud/infrastructure

Tools

Kubernetes

Job description

Cerebras Systems is seeking an Engineering Manager, Inference Router to lead the team that directs where every inference request runs across Cerebras data centers and partners. You will own global routing, cache-aware scheduling, and autoscaling, partnering with ML and cloud teams to optimize latency, throughput, and reliability.

You will build and grow a distributed engineering team, drive architectural decisions, and ensure high-availability, workload resilience, and efficient use of

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Engineering Manager, Inference Router
Engineering Manager, Inference Router

Cerebras • Toronto

On-site
CAD 180,000 - 260,000
Staff Software Engineer, AI Inference Systems
Staff Software Engineer, AI Inference Systems

Cerebras • Toronto

On-site
CAD 140,000 - 190,000
Software Engineer, Inference Platform
Software Engineer, Inference Platform

Cerebras • Toronto

On-site
CAD 256,000 - 370,000
Senior Inference ML Engineer: Transformers & Vision
Senior Inference ML Engineer: Transformers & Vision

We Love AI Jobs • Ottawa

Hybrid
CAD 180,000 - 240,000
Staff Software Engineer, Inference Platform
Staff Software Engineer, Inference Platform

Cerebras • Toronto

On-site
CAD 293,000 - 405,000
Senior Software Engineer, ML API - AI Inference
Senior Software Engineer, ML API - AI Inference

Cerebras • Toronto

On-site
CAD 130,000 - 190,000
Remote-friendly? (not stated)
Senior SDET, Inference Platform
Senior SDET, Inference Platform

Cerebras • Toronto

On-site
CAD 199,000 - 271,000
Senior Software Engineer, ML API – AI Inference
Senior Software Engineer, ML API – AI Inference

FDE Times • Toronto

On-site
CAD 209,000 - 293,000
Principal Network Security Architect
Principal Network Security Architect

Cerebras Systems • Canada

On-site
CAD 140,000 - 210,000
Senior FE Engineer - Inference Console
Senior FE Engineer - Inference Console

Cerebras • Toronto

On-site
CAD 140,000 - 180,000