Senior Software Engineer, Agent Infrastructure

cohere; Toronto (OH); Hybrid; USD 120,000 - 150,000; Full time

Benefits offered by this job

Weekly lunch stipend

Full health and dental benefits

RRSP matching, 401(k), pension scheme

100% parental leave top-up

6 weeks of paid vacation

Remote-friendly work options

Job summary

Cohere is looking for a dedicated individual to help build next-generation agentic AI infrastructure. You will work on production-ready systems with a focus on secure execution, identity management, and resource orchestration. The ideal candidate has a strong background in ML infrastructure and distributed systems, and you will collaborate closely with research teams to develop innovative solutions.

Join a passionate team at Cohere and contribute to the future of AI in a supportive environment with an array of employee benefits.

Qualifications

• Experience building production ML infrastructure.
• Hands-on work with agentic systems and multi-agent workflows.
• Familiarity with model routing and LLM frameworks.

Responsibilities

• Turn emerging ML research into production-ready infrastructure.
• Build core platform capabilities for execution and state management.
• Prototype and evaluate new technologies.

Skills

Building production ML infrastructure

Experience with agentic systems

Strong systems fundamentals

Scalable distributed systems

Tools

Kubernetes

Code execution sandboxes

Job description

Cohere is the leading security‑first enterprise AI company. We build cutting‑edge foundation AI models and end‑to‑end products that solve real‑world business problems.

We’re training and deploying frontier models for enterprises building AI systems. Our work is instrumental to the widespread adoption of AI, and we are looking for people who want to be part of that.

We obsess over what we build. Each team member contributes to increasing our models’ capabilities and the value they deliver to customers. Cohere is a team of researchers, engineers, designers, and more, all passionate about their craft.

We are a global technology company co‑headquartered in Toronto and San Francisco, with key offices in London, New York City, Montreal, Seoul, Germany, and Paris.

About the role

We’re building the next generation of agentic AI infrastructure at Cohere. This team sits at the intersection of ML systems, distributed infrastructure, and developer experience, creating the platform that powers autonomous AI agents at scale.

You’ll work on hard, forward‑looking problems with few established patterns, including secure code execution, agent state management, model routing, identity and authentication, and resource management for long‑running agent workflows.

This role is a strong fit for someone who combines systems depth with ML intuition. You should be comfortable building reliable infrastructure, thinking through distributed systems tradeoffs, and understanding how emerging agentic capabilities shape platform design.

What you’ll work on

• Secure execution environments for agent‑generated code
• Identity, authentication, and trust boundaries for agents
• Model routing and orchestration across different model types and environments
• Rate limiting, quotas, and resource management for agent workflows
• State management, memory, and filesystem abstractions for agents

In this role you will

• Turn emerging ML research ideas into production‑ready infrastructure
• Build core platform capabilities for execution, storage, and state management
• Prototype and evaluate new technologies, then help decide what should move into production
• Partner with research teams to shape infrastructure based on what future agent systems will need

You may be a good fit if you have

• Experience building production ML infrastructure with strong systems fundamentals
• Hands‑on work with agentic systems, multi‑agent workflows, or agent development frameworks
• Familiarity with model routing and LLM provider frameworks across different model types and environments
• Experience with scalable, fault‑tolerant distributed systems and Kubernetes
• A track record of moving quickly on prototypes and making good decisions about productionisation

Bonus

• Experience across on‑prem, private cloud, and public cloud environments
• Familiarity with storage systems, embedded databases, or filesystem abstractions
• Experience with code execution sandboxes such as gVisor, Firecracker, Kata, or WASM runtimes
• Interest in emerging ML infrastructure, edge inference, or browser‑native models
• Open‑source contributions to LLM or agent infrastructure projects
• Experience with identity, workload auth, or capability‑based security systems

Full‑Time Employees at Cohere enjoy these perks

• A weekly lunch stipend of $75 or £75 equivalent in your local currency
• Full health and dental benefits, including a mental health budget
• RRSP matching, 401(k), pension scheme
• 100% parental leave top‑up for up to 6 months for either parent
• Annual enrichment benefits: arts & culture, fitness/wellness, quality time, and a workspace improvement credit
• Education & learning stipend for conferences, courses, and coaching
• 6 weeks of paid vacation (30 working days)
• Budget for traveling to other offices if you are remote, plus an annual company off‑site
• Remote‑friendly with offices in Toronto, San Francisco, New York City, London, Paris, Montreal, and more coming soon
• For office workers: daily lunch program, snacks, and community events
• For remote workers: co‑working benefit and a $500 home‑office stipend

Inclusion

We strive to create an inclusive work environment for all; we welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, submit an accommodations request form and we will work together to meet your needs.