SRE for AI Platform & ML Inference Infra

Cohere; New York (NY); Hybrid; USD 140,000 - 200,000; Full time

Benefits offered by this job

Weekly lunch stipend

Health and dental benefits

RRSP matching

Parental leave

Education stipend

Vacation

Job summary

Cohere is seeking a Site Reliability Engineer to join the Model Serving team. You will build and operate high-performance, scalable AI platform components delivering large language models through reliable API endpoints.

You will work with cross-functional teams to deploy NLP models to production in low-latency, high-throughput environments, participate in on-call rotations, and help shape the infrastructure roadmap while mentoring teammates.

Qualifications

• 5+ years of engineering experience running production infrastructure at a large scale.
• Experience designing large, highly available distributed systems with Kubernetes and GPU workloads.
• Experience in designing, deploying, supporting, and troubleshooting Linux-based computing environments.

Responsibilities

• Build self-service systems that automate managing, deploying and operating services.
• Automate environment observability and resilience. Enable developers to troubleshoot and resolve problems.
• Hit defined SLOs, including participation in an on-call rotation.

Skills

Kubernetes

Distributed systems

Golang

C++

Linux

Cloud platforms

Tools

Kubernetes operators

Cloud providers (GCP, AWS, Azure, OCI)

Job description

Cohere is seeking a Site Reliability Engineer to join the Model Serving team. You will build and operate high-performance, scalable AI platform components delivering large language models through reliable API endpoints.

You will work with cross-functional teams to deploy NLP models to production in low-latency, high-throughput environments, participate in on-call rotations, and help shape the infrastructure roadmap while mentoring teammates.