Member of Technical Staff, Training Performance Engineer

Cohere; Quebec; Hybrid; CAD 250,000 - 535,000; Full time

Benefits offered by this job

Lunch stipend

Health and dental benefits

RRSP matching

Parental leave

Enrichment benefits

Annual company offsite

Home office stipend

Job summary

Cohere, a leading security-first enterprise AI company, is hiring a Performance Engineer for the Pre-Training team. You will optimize the performance of language models and systems, focusing on throughput and accelerator utilization, collaborating with researchers and engineers.

The role blends software engineering, ML, and low-level kernel design. You will write CUDA and Triton kernels, prototype training tools, and help push Cohere's capabilities in natural language understanding and

Qualifications

• Strong software engineering skills.
• Proficiency in Python and ML frameworks (JAX, PyTorch, XLA/MLIR).
• Experience writing GPU kernels with CUDA and Triton.
• Experience with large-scale distributed training.

Responsibilities

• Design and write high-performant and scalable training software.
• Understand architectural modifications and their effect on throughput.
• Write low-level CUDA, Triton kernels to maximize accelerator performance.
• Research, implement, and experiment with ideas on supercompute/data infrastructure.
• Collaborate with researchers to advance language model capabilities.

Skills

Software engineering

Python

JAX

PyTorch

XLA/MLIR

CUDA kernels

Distributed training

Transformers

Tools

CUDA

Triton

MLIR

Job description

Who are we?

Cohere is the leading security-first enterprise AI company. We buildcutting-edge foundation AI models and end-to-end products that are designed tosolve real-world business problems.

We’re training and deploying frontier models for enterprises who are building AIsystems. We believe that our work is instrumental to the widespread adoption ofAI and we are looking for folks that want to be part of that.

We obsess over what we build. Each one of us is responsible for contributing toincreasing the capabilities of our models and the value they drive for ourcustomers. Cohere is a team of researchers, engineers, designers, and more, whoare all passionate about their craft.

We are a global technology company headquartered in Toronto with key offices inLondon, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Joinus!

Role Overview

As a Performance Engineer in the Pre-Training team you will be responsible foroptimizing the performance of our advanced language models and systems. Theirprimary focus is on improving key model training metrics, such as trainingthroughput, ensuring high accelerator utilization.

The team combines expertise in software engineering, machine learning, andlow-level kernel design and development to design robust systems and enhancemodel performance. You will work on identifying and removing performancebottlenecks, develop cutting-edge training and profiling tools to help Cohere'smission of providing efficient and reliable language understanding andgeneration capabilities and drive innovation in the field of natural languageprocessing.

Note: We have offices in London, Toronto, New York and San Francisco, but we’realso remote-friendly! This team operates primarily between ET to CET time zones,so we’re seeking candidates in locations that align with these hours foreffective collaboration.

Key Responsibilities

• Design and write high-performant and scalable software for training.
• Understand architectural modifications and design choices and their effectson training throughput and quality.
• Write low-level CUDA, triton kernels to squeeze every last bit of performancefrom our accelerators.
• Research, implement, and experiment with ideas on our supercompute and datainfrastructure.
• Learn from and work with the best researchers in the field.

Qualifications

• Extremely strong software engineering skills.
• Proficiency in Python and related ML frameworks such as JAX, Pytorch andXLA/MLIR.
• Experience writing kernels for GPUs using CUDA, triton, etc
• Experience using large-scale distributed training strategies.
• Familiarity with autoregressive sequence models, such as Transformers.

Bonus: paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys,JMLR, AAAI, Nature, COLING, ACL, EMNLP).

Working location:

This role can be based remotely or from one of our office locations listed onthe job description - there is no minimum in-office qualification requirement.We care most about hiring exceptional people regardless of locations, thoughplease check the location listed on the posting for guidance around the coretime zone or working hours alignment expected for the role.

Full-Time Employees At Cohere Enjoy These Perks

• A weekly lunch stipend of $75/£75 or equivalent in your local currency forlunch.
• Full health and dental benefits, including a separate budget for mentalhealth.
• RRSP matching, 401K, Pension Scheme.
• 100% Parental Leave top-up for up to 6 months, for either parent.
• Annual enrichment benefits:
• Arts & culture, fitness/wellness, quality time, and a workspace improvementcredit.
• Education & learning stipend for conferences, courses, and coaching.
• 6 weeks of paid vacation (30 working days!)
• Budget for traveling to other offices if you are remote, plus an annualcompany offsite.

How And Where We Work

• Cohere is remote-friendly, but we also have offices in Toronto, London, NewYork City, San Francisco, Montreal, Paris, Berlin and Seoul with more openingsoon.
• For those in the office: a daily lunch program, plenty of snacks, and regularcommunity and social events.
• For those not near an office: a co-working benefit so you can work alongsideothers in your city.
• Everyone receives a $500 home office stipend to set up your workspaceproperly.

If any of the above doesn’t line up exactly with your experience, we stillencourage you to apply.

We strive to create an inclusive work environment for all; we welcome applicantsfrom all backgrounds and are committed to providing equal opportunities. Shouldyou require any accommodations during the recruitment process, please submit anAccommodations Request Form, and we will work together to meet your needs.

Compensation Range: CA$250K - CA$535K