Staff ML Engineer: Post-Training & Production

Cohere; New York (NY); Remote; USD 150,000 - 210,000; Full time

Benefits offered by this job

Weekly lunch stipend

Health and dental benefits

RRSP matching / 401K

Parental Leave top-up

Education & learning stipend

Paid vacation

Travel to other offices

Home office stipend

Job summary

Cohere is a leading security-first enterprise AI company, building foundation models and end-to-end products for real-world business problems. The role advances state-of-the-art model post-training and bridges research with production, leveraging substantial compute and talent across teams.

We value collaboration across research and engineering, offering remote-friendly arrangements and multiple office options.

Qualifications

• Extremely strong software engineering skills.
• Proficiency in Python and ML frameworks such as JAX, PyTorch and XLA/MLIR.
• Experience with distributed training infrastructures (Kubernetes, Slurm) and frameworks (Ray).
• Experience using large-scale distributed training strategies.
• Hands-on experience training large models at scale.
• Hands-on experience post-training phase with a focus on performance optimization.
• Bonus: papers at top venues (NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP).

Responsibilities

• Design and write high-performant, scalable software for training models.
• Post-train models to achieve SOTA performance.
• Coordinate with specialist teams to ensure comprehensive performance.
• Develop techniques to improve training cycles performance (SFT & RL).
• Research and experiment with ideas on compute and data infrastructure.
• Learn from and collaborate with leading researchers.

Skills

Python

JAX

PyTorch

Kubernetes

Slurm

Ray

Distributed training

Model post-training

Tools

XLA/MLIR

Large-scale training

Job description

Cohere is a leading security-first enterprise AI company, building foundation models and end-to-end products for real-world business problems. The role advances state-of-the-art model post-training and bridges research with production, leveraging substantial compute and talent across teams.

We value collaboration across research and engineering, offering remote-friendly arrangements and multiple office options.