Staff Forward-Deployed AI Engineer: Enterprise Inference

Cohere; Oregon; Hybrid; USD 180,000 - 260,000; Full time

Benefits offered by this job

Lunch stipend

Health and dental benefits

RRSP matching / 401K

Parental leave top-up

Enrichment benefits

6 weeks of paid vacation

Remote-friendly with home office stip

Job summary

Cohere is seeking a Staff Forward Deployed Engineer to bring our frontier AI models into enterprise environments. You will partner with customer engineering teams to design high-performance, secure model serving across multi-cluster and multi-cloud deployments.

You will own the architecture for ambiguous inference challenges, optimize latency and throughput, and drive production-readiness with robust observability.

Qualifications

• 8+ years of engineering experience in production infrastructure or distributed systems.
• Deep Kubernetes expertise across multi-cluster and hybrid/multi-cloud setups.
• Strong understanding of LLM architectures and inference dynamics.
• Hands-on experience with modern inference engines and optimization stacks.
• Familiarity with distributed serving techniques, such as prefill/decode disaggregated serving, distributed KV caching, and multi-GPU workloads.
• Exceptional customer-facing technical leadership with a track record of steering executives.
• The ability to write production code under deployment constraints.
• Strong Python, Go, or C++ programming foundations.

Responsibilities

• Embed with customer teams to connect AI models to live production environments.
• Design multi-tenant model serving topologies and cross-datacenter strategies for multi-cluster environments.
• Optimize private deployments balancing SRE constraints and model performance.
• Benchmark and profile LLM workloads, identify and root-cause performance bottlenecks.
• Assist with hardware validation on GPUs, TPUs, and accelerators.
• Translate field feedback into platform improvements and drive feature development.
• Ensure Day-1 production reliability and establish deployment patterns and observability.

Skills

Kubernetes

LLM Inference

Distributed Systems

Python

Go

C++

Customer Leadership

Multi-cloud

Performance Optimization

Tools

vLLM

TensorRT-LLM

LMD

Dynamo

Job description

Cohere is seeking a Staff Forward Deployed Engineer to bring our frontier AI models into enterprise environments. You will partner with customer engineering teams to design high-performance, secure model serving across multi-cluster and multi-cloud deployments.

You will own the architecture for ambiguous inference challenges, optimize latency and throughput, and drive production-readiness with robust observability.