Training Infra Engineer for Scalable AI Models

Cohere Inc.; New York, Northern (NY, KY); Hybrid; USD 160,000 - 230,000; Full time

Benefits offered by this job

Lunch stipend

Health and dental benefits

RRSP/401K match

Parental leave top-up

Enrichment benefits

Education & learning stipend

Annual offsite

Six weeks paid vacation

Home office stipend

Coworking allowance

Job summary

Cohere is seeking a highly skilled engineer to join our role-focused team, contributing to model training pipelines and production-grade code alongside researchers. You will help push frontier models from research into scalable, production-ready systems with ample compute and data resources.

We welcome remote applicants and offer a global, collaborative environment with strong emphasis on training infrastructure, tooling, and performance improvements across distributed systems.

Qualifications

• Extremely strong software engineering skills.
• Proficiency in Python and ML frameworks (JAX, PyTorch, XLA/MLIR).
• Experience with distributed training infrastructures (Kubernetes, Slurm) and Ray.
• Experience with large-scale distributed training strategies.
• Hands-on experience training large models and contributing to training tooling.

Responsibilities

• Design and write high-performant and scalable software for training.
• Improve our training setup from an infrastructure and codebase performance standpoint.
• Craft and implement tools to speed up our training cycles and improve the overall efficacy of our training infrastructure
• Research, implement, and experiment with ideas on our supercompute and data infrastructure.
• Learn from and work with the best researchers in the field.

Skills

Strong software engineering

Python & ML frameworks

Distributed training infra

Large-scale training

Research tooling experience

Top-tier paper (bonus)

Tools

Kubernetes

Slurm

Ray

Job description

Cohere is seeking a highly skilled engineer to join our role-focused team, contributing to model training pipelines and production-grade code alongside researchers. You will help push frontier models from research into scalable, production-ready systems with ample compute and data resources.

We welcome remote applicants and offer a global, collaborative environment with strong emphasis on training infrastructure, tooling, and performance improvements across distributed systems.