Staff ML Engineer: Post-Training & Production Scale

Cohere; United States; Remote; USD 180,000 - 240,000; Full time

Benefits offered by this job

An open and inclusive culture and work

Work closely with a team on the cuting

Weekly lunch stipend, in-office meals

Full health and dental benefits, incl.

100% Parental Leave top-up for 6 meses

Personal enrichment benefits for arts,

Remote-flexible, offices in Toronto,NY

6 weeks of vacation

Job summary

Cohere, a leading AI company, is seeking a Member of Technical Staff to design and write high-performing software for training models and post-train to reach SOTA level performance. You will coordinate with expert teams to deliver robust models and contribute to scalable infrastructure across supercompute and data systems.

You will learn from world-class researchers and apply innovative techniques in distributed training, with remote-friendly work options and opportunities to impact

Qualifications

• Strong software engineering skills
• Proficiency in Python and ML frameworks (JAX, PyTorch, XLA/MLIR)
• Experience with distributed training infrastructures (Kubernetes, Slurm) and Ray
• Experience using large-scale distributed training strategies
• Hands-on experience training large models at scale
• Hands-on in post-training phase with emphasis on performance optimization
• Bonus: publications at top venues (NeurIPS, ICML, ICLR, etc.)

Responsibilities

• Design and write high-performant and scalable software for training models.
• Consistently post-train models to reach SOTA level performance.
• Coordinate with specialist teams to produce models with strong overall performance.
• Craft and implement techniques to improve performance of training cycles (SFT and RL).
• Research, implement, and experiment with ideas on supercompute and data infrastructure.
• Learn from and work with leading researchers in the field.

Skills

Software engineering

Python

JAX

PyTorch

XLA/MLIR

Distributed training

Large-scale training

Post-training optimization

Research publications

Tools

Kubernetes

Slurm

Ray

Job description

Cohere, a leading AI company, is seeking a Member of Technical Staff to design and write high-performing software for training models and post-train to reach SOTA level performance. You will coordinate with expert teams to deliver robust models and contribute to scalable infrastructure across supercompute and data systems.

You will learn from world-class researchers and apply innovative techniques in distributed training, with remote-friendly work options and opportunities to impact