machine learning engineer for voice conversion

Cantina Labs; United States; Remote; USD 180,000 - 260,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

Generous equity

Paid time off and holidays

401(k) retirement plan

Lunch and snacks in-office

One Medical membership

Job summary

Cantina Labs is seeking an expert ML engineer to architect and train large-scale speech models for its AI character creator platform. You will design experiments, implement diffusion/flow-matching architectures, and optimize data pipelines and tooling for production readiness.

Responsibilities include end-to-end model development, evaluation, safety guardrails, and collaboration across data, ML, and platform teams to deliver robust, scalable audio generation capabilities.

Qualifications

• Must have exceptional research or development experience with large-scale audio models (>8B parameters, >500k hours of data).
• Experience with diffusion and/or flow-matching transformers, including samplers, schedules, conditioning mechanisms, and distillation.
• Experience training audio VAEs, neural audio codecs, and vocoders, including latent/tokenizer design and objectives.

Responsibilities

• Architect, implement, pre-train, fine-tune, and post-train or align large-scale speech models.
• Design, run, and analyze experiments to advance model understanding.
• Develop and improve tools to enhance team productivity.
• Contribute across the stack from low-level optimizations to model design.
• Define data requirements and collaborate on data curation, augmentation, labeling quality, and synthetic data strategies.
• Design automated evaluations, including listening tests and ASR-based metrics, robustness and bias checks.
• Harden the training and inference pipeline; monitor latency, memory, and cost; meet production SLAs with robust monitoring.
• Contribute to safety and consent guardrails to mitigate misuse of speech technology.

Skills

Diffusion models

Flow-matching transformers

Audio VAEs

Neural audio codecs

Vocoders

Multi-node distributed training

PyTorch

CUDA/C++

Production-ready coding

Tools

FSDP/DeepSpeed

CUDA

Triton

Python

Job description

Описание

Cantina is a social platform featuring an AI character creator. Its lifelike social characters interact across voice, video, and text, enabling users to create characters and share personalized content and group chats.

Задачи

• Architect, implement, pre-train, fine-tune, and post-train or align large-scale speech models
• Design, run, and analyze scientific experiments to advance understanding of models
• Develop and improve tools that enhance team productivity
• Contribute across the stack, from low-level optimizations to high-level model design
• Define data requirements and collaborate on data acquisition, curation, augmentation, labeling quality, and synthetic data strategies
• Design automated objective and subjective evaluations, including listening tests, SV/WER/ASR-based metrics, robustness and bias checks, and red-team studies
• Harden the training, evaluation, and inference pipeline; profile latency, memory, and cost; and meet production SLAs with robust monitoring and rollback
• Contribute to safety and consent guardrails and mitigate misuse and abuse of speech technology

Требования

• Exceptional research or development experience with large-scale audio models (>8B parameters, >500k hours of data)
• Deep hands-on experience with diffusion and/or flow-matching transformers, including samplers, schedules, conditioning mechanisms, and distillation
• Deep hands-on experience training audio VAEs, neural audio codecs, and vocoders, including latent/tokenizer design, reconstruction and perceptual objectives, and adversarial training
• Strong experience with multi-node, multi-GPU distributed training (FSDP/DeepSpeed or equivalent)
• Strong software engineering skills and a proven track record of building complex systems
• Strong PyTorch skills, performance profiling, CUDA/Triton/C++ as needed, and production-quality coding
• Experience shipping large-scale speech/audio or multimodal generative models to production
• Background working with large-scale ML data and ability to iterate on data and assess quality using subjective and objective signals
• Experience with voice cloning, speech control/steerability, or expressive speech generation
• Notable publications and/or open-source contributions in speech/audio/ML

Условия

• Competitive salary and generous company equity
• Medical, dental, and vision insurance, with 99.99% of premiums covered by Cantina
• 42 Days of paid time off: 15 PTO days, 10 sick days, 15 company holidays, and 2 floating holidays
• Generous parental leave and fertility support
• 401(K) retirement savings plan
• Lifestyle spending account of $500/month
• Complimentary lunch and snacks for in-office employees
• One Medical membership and other benefits