Remote Audio Inference Engineer — Fast ML Serving

Visa Hunt; New York (NY); Hybrid; USD 140,000 - 190,000; Full time

Benefits offered by this job

Weekly lunch stipend

Health and dental benefits

RRSP matching / 401K / Pension

Parental Leave

Enrichment benefits

Education stipend

Six weeks vacation

Travel budget to other offices

Home office stipend

Job summary

Cohere in New York is seeking an experienced ML systems engineer to advance high‑performance audio and streaming inference on real-time workloads. You will optimize latency, throughput, and quality while collaborating with training and serving teams to ensure seamless model deployment.

Proficiency in C++, Python, and deep learning frameworks (PyTorch or TensorFlow) is essential, with hands‑on experience in audio, speech, or language models.

Qualifications

• Significant experience developing high-performance audio or machine learning inference systems.
• Proficiency with programming languages such as C++ and Python.
• Hands-on experience with deep learning models for audio, speech, or language applications.
• A bias for action and a strong results-oriented mindset.

Responsibilities

• Build reliable audio inference systems and optimize latency, throughput, and quality.
• Collaborate with both training and serving infrastructure teams to ensure seamless integration.
• Dive deep into systems, identify bottlenecks, and deliver creative streaming solutions for audio workloads.
• Contribute to real-time and streaming audio model serving improvements.

Skills

C++

Python

Audio ML

GPU programming

Real-time streaming

PyTorch

TensorFlow

SGLang

vLLM

Distributed inference

Tools

PyTorch

TensorFlow

Job description

Cohere in New York is seeking an experienced ML systems engineer to advance high‑performance audio and streaming inference on real-time workloads. You will optimize latency, throughput, and quality while collaborating with training and serving teams to ensure seamless model deployment.

Proficiency in C++, Python, and deep learning frameworks (PyTorch or TensorFlow) is essential, with hands‑on experience in audio, speech, or language models.