Multimodal Speech ML Engineer - Real-Time AV AI

Cantina; United States; Remote; USD 200,000 - 220,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

Equity

Paid time off: 42 days including PTO,

Parental leave & fertility support

401(k) retirement plan

Lifestyle spending account

Office lunch and snacks

Job summary

Cantina Labs is hiring a Research / ML Engineer to join our Speech Team and own the audio side of multimodal generation. You will design, train, and deploy audio VAEs, neural codecs, and diffusion-based backbones, with multi-speaker conditioning and AV synchronization for cinematic dialogue and sound design.

You will collaborate across research, video, data, and infra to ship reliable, cost-aware models; lead experiments, tune data pipelines, and push safe, steerable AI that speaks and emotes in

Qualifications

• Extensive research/development experience with large-scale audio models and multimodal systems.
• Hands-on diffusion/flow-matching transformer work and practical audio/VAE/codec knowledge.
• Experience with multi-node distributed training and production-grade code.

Responsibilities

• Design, train, and improve audio VAEs, neural codecs, and vocoders for large-scale AV models.
• Architect and fine-tune diffusion/flow-matching transformers for audio/video generation.
• Develop audio conditioning and cross-modal alignment in joint AV models.
• Run experiments, analyze results, and drive data-driven improvements.
• Own data requirements, curation, AV-sync, and quality filtering for paired corpora.
• Evaluate audio fidelity, intelligibility, AV-sync, and robustness; address safety concerns.

Skills

Audio modeling

Multimodal analysis

Diffusion models

PyTorch

Distributed training

Voice cloning

Video generation

Experimentation

Data collaboration

Tools

CUDA

DeepSpeed

TensorRT

Job description

Cantina Labs is hiring a Research / ML Engineer to join our Speech Team and own the audio side of multimodal generation. You will design, train, and deploy audio VAEs, neural codecs, and diffusion-based backbones, with multi-speaker conditioning and AV synchronization for cinematic dialogue and sound design.

You will collaborate across research, video, data, and infra to ship reliable, cost-aware models; lead experiments, tune data pipelines, and push safe, steerable AI that speaks and emotes in