Staff Engineer, Production AI & Scalable LLMs

cohere; New York (NY); On-site; USD 150,000 - 230,000; Full time

Benefits offered by this job

Lunch stipend

Health benefits

RRSP/401K

Parental leave

Learning stipend

Paid vacation

Office travel

Job summary

Cohere is seeking a Senior Engineer to design and implement scalable training systems for frontier AI models. The role focuses on production-ready software, tooling for research, and cross-team collaboration to bridge research with practical deployment.

You will work on distributed training environments, leverage Kubernetes, Ray, and Python ML frameworks, and contribute to ML/LLM research. This is a remote-friendly, production-oriented position with generous perks.

Qualifications

• Proficiency in Python and ML frameworks (JAX, PyTorch, XLA/MLIR).
• Experience with large-scale distributed training strategies (memory/speed profiling).
• Experience with distributed training infrastructures (Kubernetes) and Ray.
• Experience in ML and LLM academic research.
• Strong software engineering and test-driven development practices.

Responsibilities

• Design and write high-performing, scalable software for training models.
• Develop new tools to support and accelerate research and LLM training.
• Coordinate with Infrastructure, Efficiency, Serving teams and the scientific teams.
• Research, implement, and experiment with ideas on our cluster and data infrastructure.
• Collaborate with scientists, engineers, and teams across the company.

Skills

Strong software engineering

Test-driven development

Python

ML frameworks (JAX, PyTorch, XLA/MLIR)

Distributed training (memory/speed)

Kubernetes

Ray

ML & LLM research

Tools

Kubernetes

Ray

Job description

Cohere is seeking a Senior Engineer to design and implement scalable training systems for frontier AI models. The role focuses on production-ready software, tooling for research, and cross-team collaboration to bridge research with practical deployment.

You will work on distributed training environments, leverage Kubernetes, Ray, and Python ML frameworks, and contribute to ML/LLM research. This is a remote-friendly, production-oriented position with generous perks.