Senior Staff Engineer, Model Efficiency - Remote

Cohere; United States; Remote; USD 150,000 - 210,000; Full time

Job summary

Cohere is seeking an engineer to build reliable ML systems and optimize LLM inference for enterprise-grade AI workloads. You will dive into the inference stack to reduce latency, improve throughput, and ensure quality across diverse tasks.

The role involves collaborating with modeling and systems teams, exploring GPU/CUDA optimizations, and implementing kernel-level improvements for large-scale architectures.

Qualifications

• 5+ years of experience writing high-performance, production-quality code.
• Strong programming skills in C++ or Python (Rust/Go also welcome).
• Experience with large language models and familiarity with the LLM inference ecosystem.

Responsibilities

• Improve core performance metrics by optimizing model execution and bottlenecks.
• Collaborate with modeling and systems teams to measure and ship improvements.
• Work across the inference stack to reduce latency and increase throughput.

Skills

C++

Python

Large language models

Performance optimization

Job description

Cohere is seeking an engineer to build reliable ML systems and optimize LLM inference for enterprise-grade AI workloads. You will dive into the inference stack to reduce latency, improve throughput, and ensure quality across diverse tasks.

The role involves collaborating with modeling and systems teams, exploring GPU/CUDA optimizations, and implementing kernel-level improvements for large-scale architectures.