Machine Learning Engineer, Speech LLMs

DeepRec.ai; San Francisco (CA); Hybrid; USD 243,000 - 297,000; Full time

Benefits offered by this job

Performance bonus

Equity

Healthcare for you and dependents

401(k) with company matching

Job summary

DeepRec.ai is building a core SpeechLLM lab in San Francisco and is seeking an early member to help train models that understand and generate speech natively. You will work across the stack from signal processing to foundation model training and take ownership of uncertain problems, driving production-ready solutions.

The role requires deep proficiency in PyTorch or JAX, optimizing distributed training, and a strong interest in AI systems that handle speech directly.

Qualifications

• Proven track record building large-scale audio or speech models.
• Deep expertise in PyTorch or JAX, with experience in optimizing distributed training and managing GPU memory.
• Comfort moving between research and engineering and interest in AI systems that handle speech natively.

Responsibilities

• Build and train large-scale audio/speech models from the ground up across the full stack from signal processing to foundation model training.
• Design and evaluate new sequence modelling architectures and debugging distributed training clusters.
• Own ambiguous problems and bring solutions into production.

Skills

PyTorch

JAX

Distributed training

GPU memory

Research to Engineering

Tools

vLLM

TensorRT-LLM

SGLang

DeepSpeed

Kubernetes

Job description

You’d be an early member of a new speech LLM lab inside a profitable, bootstrapped company, with the job of training models that understand and generate speech natively.

About the company

A profitable Voice AI company based in San Francisco. It builds hardware and software that capture what professionals say and hear, then turn it into notes and usable information. The business has a large global user base and is now setting up a core SpeechLLM lab, which this person would join as a founding hire.

The role

You’ll build and train large-scale audio and speech models from the ground up. That could mean unified SpeechLLMs, advanced ASR, expressive TTS or generative audio architectures. One day you’re designing a new sequence modelling architecture, the next you’re debugging a distributed training cluster. The work runs across the whole stack, from signal processing and raw acoustic representations to foundation model training and optimisation for edge devices. You’ll own ambiguous problems and take them into production.

What we’re looking for

A proven track record of building and training large-scale audio or speech models. Deep expertise in PyTorch or JAX, including optimising large distributed training runs, managing GPU memory and fixing performance bottlenecks. Comfort moving between research and engineering, and a real interest in AI systems that handle speech natively.

Any of the following is a bonus:

• Text LLM pretraining, instruction tuning or RLHF
• Neural audio codecs for streamable, high-fidelity audio
• Diffusion, flow matching or autoregressive models for speech and voice generation
• RL techniques such as RLHF or GRPO to improve conversational cadence, steerability and alignment
• Inference optimisation with serving frameworks like vLLM, TensorRT-LLM or SGLang
• Large GPU clusters, distributed training with FSDP or DeepSpeed, and Kubernetes

Salary and benefits

• Up to $270K, plus a performance bonus and equity.
• Healthcare for you and your dependents, including dental and vision, with a generous employer subsidy.
• 401(k) with company matching.

Logistics

Full-time, permanent. Based in San Francisco, hybrid, with a minimum of three days a week in the office.