Lead ML Infra Architect for High-Performance NLP

Cohere; New York (NY); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Weekly lunch stipend

Health and dental benefits

RRSP matching / 401K / Pension

Parental leave top-up

Enrichment benefits & learning stipend

6 weeks paid vacation

Travel budget to other offices / off‑s

Home office stipend

Job summary

Cohere is seeking a Lead Member of Technical Staff to drive the Model Serving platform, delivering low-latency NLP model deployments via scalable, reliable APIs. You will mentor engineers, define architecture direction, and coordinate across teams to meet customers’ needs.

The role focuses on leading design and deployment of high-performance, distributed ML infrastructure with strong emphasis on Kubernetes, cloud platforms, and accelerator technologies.

Qualifications

• 8+ years of engineering leadership running production infrastructure at scale.
• Architecture and design of large, highly available distributed systems on Kubernetes with GPU workloads.
• Deep expertise with Kubernetes development and production standards.
• Experience across multi-cloud (GCP, Azure, AWS, OCI) and on-prem/hybrid environments.
• Leading design, deployment, and troubleshooting of Linux-based computing environments at scale.
• Ownership of compute/storage/network resources and cost management at an organizational level.
• Strong collaboration and mentoring skills across cross-functional teams.
• GPGPU/accelerator knowledge to optimize latency and throughput at scale.
• Proficiency in high-performance servers using Golang and C++.

Skills

Engineering leadership

Kubernetes

Distributed systems

GPU workloads

Cloud platforms

Linux

Cost management

Mentoring

Adaptability

GPUs (accelerators)

C++

Golang

High-performance servers

Job description

Cohere is seeking a Lead Member of Technical Staff to drive the Model Serving platform, delivering low-latency NLP model deployments via scalable, reliable APIs. You will mentor engineers, define architecture direction, and coordinate across teams to meet customers’ needs.

The role focuses on leading design and deployment of high-performance, distributed ML infrastructure with strong emphasis on Kubernetes, cloud platforms, and accelerator technologies.