GPU Infra Staff Engineer — Multi-Cloud Kubernetes

Cohere; United States; Remote; USD 180,000 - 240,000; Full time

Benefits offered by this job

Lunch stipend

Health and dental benefits

RRSP matching/401K/Pension

6 weeks paid vacation

Parental leave top-up

Home office stipend

Job summary

Cohere is seeking to strengthen its Internal Infrastructure team. You will deploy and operate GPU Kubernetes clusters across clouds, work with cloud providers and hardware partners on reliability and performance, and automate fault detection so most issues are fixed without human intervention.

You will also focus on fleet utilization, cost tracking, and rapid recovery of failed nodes while collaborating with AI researchers and ML engineers to keep Cohere’s foundation models performant and

Qualifications

• Proven experience deploying and operating GPU Kubernetes clusters across clouds.
• Strong debugging skills in Linux, networking, and performance issues.
• Proficiency in Python or Go and contributing to open-source projects.
• Experience collaborating with AI researchers or ML engineers.
• Ability to identify bottlenecks and drive fixes in fast-moving environments.

Responsibilities

• Deploy and operate GPU Kubernetes clusters across clouds and bring up new hardware generations.
• Collaborate with cloud providers and hardware partners on reliability and performance.
• Automate node, GPU, and fabric fault detection and remediation to reduce human intervention.
• Improve fleet utilization, track costs, and accelerate recovery of faulty nodes.
• Troubleshoot production failures and performance issues, simplifying operations.

Skills

GPU Kubernetes clusters

Linux debugging

Python or Go

Collaboration with researchers/ML

Performance optimization

Tools

RDMA

NCCL

Job description

Cohere is seeking to strengthen its Internal Infrastructure team. You will deploy and operate GPU Kubernetes clusters across clouds, work with cloud providers and hardware partners on reliability and performance, and automate fault detection so most issues are fixed without human intervention.

You will also focus on fleet utilization, cost tracking, and rapid recovery of failed nodes while collaborating with AI researchers and ML engineers to keep Cohere’s foundation models performant and