Staff Forward-Deployed AI Inference Engineer

Cohere; United States; Hybrid; USD 180,000 - 260,000; Full time

Benefits offered by this job

Lunch stipend

Health and dental benefits

RRSP matching / 401K

Parental leave top-up

Enrichment benefits

Education stipend

6 weeks paid vacation

Travel budget to other offices

Home office stipend

Job summary

Cohere is seeking a Staff Forward Deployed Engineer to bridge frontier AI capability with complex customer environments. You will own the architecture for ambiguous inference challenges—from multi-tenant model serving to cross-datacenter disaster recovery and custom hardware validation—ensuring latency, throughput, and resilience in the field.

You will embed with customer teams, lead technical adoption end-to-end, and translate field friction into platform improvements, while working with

Qualifications

• 8+ years engineering production infrastructure experience in large-scale environments.
• Strong Kubernetes expertise across multi-cluster and multi-cloud setups (GCP, Azure, AWS, OCI).
• Deep understanding of LLM architecture, inference dynamics and live performance.
• Hands-on with modern inference engines and optimization stacks.

Responsibilities

• Embed with customer teams to connect AI models to production environments.
• Design topologies for multi-tenant serving and cross-datacenter failover.
• Optimize private deployments balancing SRE constraints and model performance.
• Benchmark and profile customer LLM workloads, diagnose slow generation.
• Support Cohere models on GPUs, TPUs, and custom accelerators.
• Translate field feedback into platform improvements and product features.
• Ensure Day-1 production reliability with clear deployment patterns and observability.

Skills

Kubernetes

LLM inference

Python

Go

C++

Multi-cluster

Distributed systems

Customer leadership

Cloud platforms

Performance optimization

Tools

vLLM

TensorRT-LLM

lmd

Dynamo

Job description

Cohere is seeking a Staff Forward Deployed Engineer to bridge frontier AI capability with complex customer environments. You will own the architecture for ambiguous inference challenges—from multi-tenant model serving to cross-datacenter disaster recovery and custom hardware validation—ensuring latency, throughput, and resilience in the field.

You will embed with customer teams, lead technical adoption end-to-end, and translate field friction into platform improvements, while working with