Staff Software Engineer, AI Inference Platform

Visa Hunt; San Francisco (CA); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Lunch stipend

Health & dental benefits

Parental leave top-up

Enrichment benefits

Vacation time

Travel to offices

Job summary

Cohere is seeking a Members of Technical Staff to join the Model Serving team. You will deploy optimized NLP models to production with low latency and high throughput, interfacing with multiple teams and customers to tailor deployments for specific needs.

The role emphasizes building scalable, reliable AI infrastructure on Kubernetes across multi-cloud environments, with GPU workloads and performance tuning in Linux-based systems. Strong collaboration and problem-solving are essential.

Qualifications

• 5+ years creating production infrastructure at scale.
• Designing large, highly available distributed systems with Kubernetes and GPUs.
• Kubernetes dev and prod coding and support experience.
• Experience with GCP, Azure, AWS, OCI, multi-cloud.
• Linux-based compute environments: design, deploy, troubleshoot.
• Compute/storage/network resource and cost management.
• Strong collaboration and troubleshooting for critical systems.
• Grit and adaptability for evolving technical challenges.
• Familiarity with GPUs/TPUs and latency/throughput effects.
• Strong distributed systems knowledge.
• Golang or C++ high-performance servers experience.

Responsibilities

• Deploy and operate scalable AI platform components.
• Collaborate across teams to tailor deployments for customers.
• Optimize latency and throughput for NLP models in production.
• Ensure high availability and reliability in multi-cloud environments.

Skills

Distributed systems

Kubernetes

Golang/C++

Multi-cloud

Linux troubleshooting

Collaboration

Latency optimization

GPU accelerators

GPUs/TPUs

Distributed systems knowledge

High-performance servers

Tools

Kubernetes

GCP

AWS

Azure

OCI

Linux

GPUs

Job description

Cohere is seeking a Members of Technical Staff to join the Model Serving team. You will deploy optimized NLP models to production with low latency and high throughput, interfacing with multiple teams and customers to tailor deployments for specific needs.

The role emphasizes building scalable, reliable AI infrastructure on Kubernetes across multi-cloud environments, with GPU workloads and performance tuning in Linux-based systems. Strong collaboration and problem-solving are essential.