Staff Software Engineer — ML Platform & Inference

Cohere; San Francisco, New York (CA, NY); Hybrid; USD 180,000 - 280,000; Full time

Benefits offered by this job

Lunch stipend

Health and dental benefits

RRSP matching / 401K

Parental Leave

Education stipend

Vacation 6 weeks

Office/remote travel budget

Home office stipend

Job summary

Cohere is seeking Members of Technical Staff to join the Model Serving team in San Francisco. You will help build and operate the AI platform delivering large language models through easy-to-use API endpoints.

Ideal candidates have 5+ years of engineering in production infrastructure, experience with Kubernetes, cloud platforms, and high-performance distributed systems. This role is remote-friendly with office options in SF and travel per policy.

Qualifications

• 5+ years of engineering experience running production infrastructure at a large scale.
• Experience designing large, highly available distributed systems with Kubernetes.
• Experience with multi-cloud deployments (GCP, Azure, AWS) and on-prem/hybrid setups.
• Proficient in Linux-based compute environments and resource management.
• Strong collaboration and troubleshooting skills for mission-critical systems.

Responsibilities

• Develop, deploy, and operate the AI platform delivering Cohere's large language models through API endpoints.
• Collaborate with multiple teams to deploy optimized NLP models to production with low latency and high throughput.
• Ensure high availability and reliability of production services across clusters and clouds.
• Interface with customers to design customized deployments meeting their needs.

Skills

Production infrastructure

Kubernetes

Distributed systems

Cloud platforms

Linux

GPU workloads

Go/C++

Performance tuning

Troubleshooting

Job description

Cohere is seeking Members of Technical Staff to join the Model Serving team in San Francisco. You will help build and operate the AI platform delivering large language models through easy-to-use API endpoints.

Ideal candidates have 5+ years of engineering in production infrastructure, experience with Kubernetes, cloud platforms, and high-performance distributed systems. This role is remote-friendly with office options in SF and travel per policy.