Senior ML Infra Engineer for Agentic AI Platform

Cohere; Toronto; Hybrid; CAD 130,000 - 180,000; Full time

Benefits offered by this job

Weekly lunch stipend

Health and dental benefits

RRSP matching / 401K

Parental leave top-up

Enrichment benefits

Vacation time (6 weeks)

Travel budget to other offices

Home office stipend

Job summary

Cohere is hiring for a role at the intersection of ML systems and infrastructure, building scalable agentive AI platforms. You’ll tackle hard problems and contribute to secure code execution, state management, and model routing at scale.

You will work on secure execution, identity and authentication boundaries, and resource management for long-running agent workflows, collaborating with research teams to shape future infrastructure.

Qualifications

• Experience building production ML infrastructure with strong systems fundamentals.
• Hands-on work with agentic systems, multi-agent workflows, or agent development frameworks.
• Familiarity with model routing and LLM provider frameworks across different model types and environments.
• Experience with scalable, fault-tolerant distributed systems and Kubernetes.
• A track record of moving quickly on prototypes and making good decisions about productionization.

Responsibilities

• Turn emerging ML research ideas into production-ready infrastructure.
• Build core platform capabilities for execution, storage, and state management.
• Prototype and evaluate new technologies, then help decide what should move into production.
• Partner with research teams to shape infrastructure based on what future agent systems will need.

Skills

Production ML infrastructure

Agent systems experience

Model routing familiarity

Distributed systems

Strong systems fundamentals

Tools

Kubernetes

gVisor

Firecracker

Kata

WASM runtimes

Job description

Cohere is hiring for a role at the intersection of ML systems and infrastructure, building scalable agentive AI platforms. You’ll tackle hard problems and contribute to secure code execution, state management, and model routing at scale.

You will work on secure execution, identity and authentication boundaries, and resource management for long-running agent workflows, collaborating with research teams to shape future infrastructure.