Staff ML Systems Engineer - Model Efficiency & Inference

Cohere; Toronto; Hybrid; CAD 140,000 - 200,000; Full time

Benefits offered by this job

Lunch stipend

Health benefits

Retirement plan matching (RRSP/401K)

Job summary

Cohere is a security-first AI company focused on building efficient foundation models and end-to-end products for enterprises. This role targets optimizing the inference stack, reducing latency, and boosting throughput for large language models.

You will work with researchers and engineers across modeling and systems teams, exploring GPU-accelerated optimizations and MoE strategies to accelerate production deployments.

Qualifications

• 5+ years of experience writing high-performance, production-quality code.
• Strong programming skills in C++ or Python (Rust/Go also welcome).
• Experience with large language models and familiarity with the LLM inference ecosystem (e.g., vLLM, SGLang, etc.)
• Ability to diagnose and resolve performance bottlenecks across the model execution stack
• A strong bias for action — you ship fast, measure impact, and iterate.

Responsibilities

• Work across the inference stack to improve core performance metrics.
• Collaborate closely with modeling and systems teams to experiment, measure, and ship improvements.
• Build expertise in advanced performance techniques, including GPU/CUDA optimizations and model execution strategies.

Skills

C++

Python

GPU programming

LLM inference

Performance analysis

Distributed systems

Tools

CUDA

Job description

Cohere is a security-first AI company focused on building efficient foundation models and end-to-end products for enterprises. This role targets optimizing the inference stack, reducing latency, and boosting throughput for large language models.

You will work with researchers and engineers across modeling and systems teams, exploring GPU-accelerated optimizations and MoE strategies to accelerate production deployments.