Senior ML Training Systems Engineer - Frameworks & Tooling

Cohere; Toronto; Hybrid; CAD 100,000 - 140,000; Full time

Job summary

A leading AI company in Toronto is seeking a Senior Engineer to build and maintain the training framework for large-scale language models. This role involves working on distributed systems and HPC infrastructure, collaborating closely with various teams to enhance training efficiency. Ideal candidates have strong engineering experience, expertise in tools like JAX and Kubernetes, and a passion for machine learning systems. The company offers a collaborative environment with significant opportunities for impact and professional growth.

Benefits

Open and inclusive culture

Weekly lunch stipend

Full health and dental benefits

100% Parental Leave top-up

Remote-flexible arrangement

6 weeks of vacation

Qualifications

• Strong engineering experience in large-scale distributed training or HPC systems.
• Familiarity with multi-node cluster orchestration.
• A track record of increasing developer velocity for ML.

Responsibilities

• Build and own the training framework for large-scale LLM training.
• Design distributed training abstractions.
• Investigate and resolve performance bottlenecks.

Skills

Large-scale distributed training

HPC systems

Debugging performance issues

Containerized environments

Building tools for ML teams

Collaboration skills

Tools

JAX

Kubernetes

Docker

Job description

A leading AI company in Toronto is seeking a Senior Engineer to build and maintain the training framework for large-scale language models. This role involves working on distributed systems and HPC infrastructure, collaborating closely with various teams to enhance training efficiency. Ideal candidates have strong engineering experience, expertise in tools like JAX and Kubernetes, and a passion for machine learning systems. The company offers a collaborative environment with significant opportunities for impact and professional growth.