Lead ML Infrastructure Engineer (Kubernetes + GPUs)

Cohere; California (MO); Hybrid; USD 180,000 - 260,000; Full time

Benefits offered by this job

Lunch stipend

Health and dental benefits

RRSP matching / 401K

Parental leave top-up

Learning stipend

Paid vacation (6 weeks)

Travel to other offices

Home office stipend ($500)

Job summary

Cohere is seeking a Lead Member of Technical Staff to drive the architecture and deployment of scalable NLP model-serving platforms. You will own the end-to-end design of low-latency, high-throughput API endpoints and guide cross-functional teams to deliver production-grade infrastructure.

You will mentor engineers, set technical direction across multiple teams, and optimize compute/storage networks for cost efficiency while supporting GPU-accelerated workloads in hybrid multi-cloud environments.

Qualifications

• 8+ years of engineering experience leading production infrastructure at scale.
• Experience architecting large, highly available distributed systems.
• Kubernetes experience in dev and production environments.
• Multi-cloud/ hybrid serving environments (GCP, Azure, AWS, OCI).
• Proven ability to lead design, deployment and operations of Linux-based computing environments at scale.
• Resource and cost management at organizational level.
• Strong collaboration and communication, mentoring engineers.
• Experience with GPUs/accelerators to improve latency and throughput.

Responsibilities

• Lead architecture and design of large, distributed systems with Kubernetes and GPU workloads.
• Own deployment, support, and troubleshooting of the AI platform for production NLP models via APIs.
• Mentor engineers and drive cross-functional initiatives to build mission-critical systems.
• Guide infrastructure decisions across multi-cloud and on-prem/hybrid environments.
• Serve as a primary technical contact for customers for customized deployments.

Skills

Engineering leadership

Distributed systems

Kubernetes

Cloud platforms

Linux systems

Cost optimization

GPU acceleration

Go / C++

Tools

Kubernetes

Golang

C++

Job description

Cohere is seeking a Lead Member of Technical Staff to drive the architecture and deployment of scalable NLP model-serving platforms. You will own the end-to-end design of low-latency, high-throughput API endpoints and guide cross-functional teams to deliver production-grade infrastructure.

You will mentor engineers, set technical direction across multiple teams, and optimize compute/storage networks for cost efficiency while supporting GPU-accelerated workloads in hybrid multi-cloud environments.