Lead ML Evaluation Engineer — Speech AI & Metrics

Cantina Labs; United States; Remote; USD 140,000 - 200,000; Full time

Benefits offered by this job

Competitive salary

Equity

Medical insurance

PTO 42 days

Parental leave

401(k)

Lunch provided

One Medical

Job summary

Cantina Labs seeks an experienced Machine Learning Engineer to lead audio model evaluation for speech generation and recognition. You will design and implement comprehensive evaluation pipelines for development and production, and create automated dashboards to report results.

As a founding member of the evaluation team, you will shape the direction of our evaluation efforts, collaborate with the model, data, and product teams, and help grow the team as Cantina expands its AI-driven storytelling

Qualifications

• Experience designing metrics that capture model performance.
• Experience designing user studies on Mechanical Turk or similar platforms.
• Experience with training and fine-tuning models for evaluation.
• Strong statistical knowledge to compare results and drive decisions.
• Very strong engineering and programming skills.

Responsibilities

• Design model evaluation pipelines for development and production.
• Design user studies for subjective model evaluations.
• Convert requirements into measurable metrics.
• Develop automated evaluation dashboards to monitor performance.
• Train new models to capture new evaluation metrics.
• Communicate evaluation results to model, data, and product teams.
• Help grow and lead the evaluation team as the founding member.

Skills

Model eval design

User studies (MTurk)

Model training & fine-tuning

Statistics

Engineering skills

ASR/TTS experience

Large-scale ML experience

Tools

ASR

TTS

Job description

Cantina Labs seeks an experienced Machine Learning Engineer to lead audio model evaluation for speech generation and recognition. You will design and implement comprehensive evaluation pipelines for development and production, and create automated dashboards to report results.

As a founding member of the evaluation team, you will shape the direction of our evaluation efforts, collaborate with the model, data, and product teams, and help grow the team as Cantina expands its AI-driven storytelling