Senior ML Infrastructure Lead — Model Serving

Cohere; United States; Hybrid; USD 210,000 - 260,000; Full time

Benefits offered by this job

Lunch stipend

Health and dental benefits

RRSP matching/401K

Parental leave top-up

Vacation 6 weeks

Travel to other offices / offsite

Home office stipend

Job summary

Cohere is seeking a Lead Member of Technical Staff to join the Model Serving team. You will provide technical leadership across multiple teams, shape deployment of optimized NLP models to production with low latency and high throughput, and mentor engineers to raise the technical bar.

You will lead architecture decisions for scalable, distributed Linux-based compute environments, manage resource optimization, and collaborate with customers to meet deployment needs across multi-cloud and on-prem

Qualifications

• 8+ years of engineering experience leading production infrastructure at scale.
• Experience leading architecture and design of large, highly available distributed systems.
• Strong Kubernetes development and production practices.
• Extensive multi-cloud experience (GCP, AWS, Azure, OCI) and on-prem/hybrid environments.
• Proven ability to design, deploy, and troubleshoot complex Linux-based compute environments.
• Experience with cost/resource management at an organizational level.

Responsibilities

• Lead the architecture and direction for deploying optimized NLP models to production.
• Provide technical leadership across multiple teams and mentor engineers.
• Define standards, reviews, and best practices for high-performance, scalable servers.
• Collaborate with customers to design customized deployments meeting specific needs.
• Drive latency and throughput improvements across GPU-accelerated workloads.
• Represent Cohere in cross-functional initiatives to build mission-critical systems.

Skills

Kubernetes

Distributed systems

Golang/C++

GPU workloads

Multi-cloud infra

Leadership

Linux

Tools

Terraform

Job description

Cohere is seeking a Lead Member of Technical Staff to join the Model Serving team. You will provide technical leadership across multiple teams, shape deployment of optimized NLP models to production with low latency and high throughput, and mentor engineers to raise the technical bar.

You will lead architecture decisions for scalable, distributed Linux-based compute environments, manage resource optimization, and collaborate with customers to meet deployment needs across multi-cloud and on-prem