Machine Learning Engineer, Core Evaluations

Cantina Labs

Netherlands

Remote

EUR 90,000 - 150,000

Full time

9 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Competitive salary
Generous equity
Insurance – medical, dental, vision
Paid time off – 42 days
Parental leave & fertility support
401(k) retirement plan
Lifestyle spending account
In-office lunch and snacks
One Medical membership

Job summary

Cantina Labs is seeking an experienced Machine Learning Engineer to lead audio model evaluation for speech generation and recognition. You will build evaluation pipelines for development and production, and create automated dashboards reporting results, while shaping the future of the evaluation team.

As a founding member, you will drive metric design, coordinate with product and data teams, and guide model improvements based on comprehensive evaluations.

Qualifications

  • Ability to design metrics that capture model performance.
  • Experience designing user studies on Mechanical Turk or similar platforms.
  • Experience with training and fine-tuning models for evaluation.
  • Strong statistical knowledge to compare evaluation results and guide decisions.
  • Very strong engineering and programming skills.
  • Experience with training ASR and TTS models.
  • Experience in ML teams handling large-scale ML problems.

Responsibilities

  • Design evaluation pipelines for models in development and production.
  • Design user studies for subjective model evaluations.
  • Convert requirements into measurable metrics.
  • Develop automated evaluation dashboards to visualize model performance.
  • Train new models to capture new metrics.
  • Collaborate with the model team to improve models based on evaluation results.
  • Collaborate with the data team on data needs to improve performance.
  • Coordinate with product managers to ensure correct measurement of requirements.
  • Help grow and lead the evaluation team as the founding member.

Skills

Metrics design
User studies (Mechanical Turk)
Model training & fine-tuning
Statistics & data analysis
Engineering & programming
ASR & TTS training
Large-scale ML experience
>3B parameter models with >1m hours of

Job description

About Cantina:

Cantina Labs is a social AI company, developing a suite of advanced real-time models that push the boundaries of expression, personality, and realism. We bring characters to life, transforming how people tell stories, connect, and create. We build and power ecosystems. Cantina, our flagship social AI platform, is just the beginning. If you're excited about the potential AI has to shape human creativity and social interactions, join us in building the future!

About the Role:

We are seeking an experienced Machine Learning Engineer (MLE) to focus on audio model evaluation, specifically for speech generation and recognition models. This role involves designing and developing comprehensive model evaluation pipelines for both development and production environments, as well as creating automated dashboards for reporting evaluation results. As the founding member of our evaluation team, the ideal candidate is expected to leverage their experience to lead our evaluation efforts and play a key role in the future growth of the evaluation team.

What You’ll Do:
  • Designing model evaluation pipelines for models in development and production
  • Designing user studies for subjective model evaluations.
  • Converting requirements into measurable metrics.
  • Designing and developing automated evaluation dashboard to see model performances and compare results.
  • Training new models to capture new and different evaluation metrics.
  • Communicating with the model team to help design better models based on the evaluation results.
  • Communicating with the data team to help decide the type of data necessary to improve model performance.
  • Communication with the product-manager to make sure product requirements are correctly measured.
  • Help grow the evaluation team as the founding member.
  • Lead the evaluation team in the future.
What You’ll Bring:
  • Strong experience and intuition for designing metrics that capture model performance.
  • Strong experience with designing user studies on Mechanical Turk or similar platforms.
  • . Strong experience with model training and fine-tuning for model evaluation.
  • Strong statistical knowledge and experience to statistically compare evaluation results and take decisions.
  • Very strong engineering and programming skills.
  • Experience with training ASR, TTS models.
  • Experience at ML teams working on large-scale machine learning problems.
  • (>3B models with >1m hours of data)
Compensation:

When determining compensation, a number of factors will be considered, including skills, experience, job scope, location, and competitive compensation market data.

Benefits for U.S.-based roles:
  • Competitive salary and generous company equity
  • Medical, dental, and vision insurance – 99.99% of premiums covered by Cantina
  • 42 days of paid time off, including: 15 PTO days 10 sick days 15 company holidays 2 floating holidays
  • Generous parental leave & fertility support
  • 401(k) retirement savings plan
  • Lifestyle spending account – $500/month to use however you’d like
  • Complimentary lunch and snacks for in-office employees
  • One Medical membership, and more!
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Machine Learning Engineer, Voice Conversion
Machine Learning Engineer, Voice Conversion

Cantina Labs • Netherlands

Remote
EUR 120,000 - 180,000
Equity
Medical/dental/vision insurance
42 days of PTO (paid time off)
+5
Research Scientist, Real-Time Video Generation
Research Scientist, Real-Time Video Generation

Cantina Labs • Netherlands

Remote
EUR 90,000 - 140,000
Competitive salary
Company equity
Medical, dental, and vision insurance
+5
Founding ML Engineer, Audio Evaluation & Pipelines
Founding ML Engineer, Audio Evaluation & Pipelines

Cantina Labs • Netherlands

Remote
EUR 90,000 - 150,000
Competitive salary
Generous equity
Insurance – medical, dental, vision
+6
AI/LLM Engineer (Model Fine-Tuning & Conversational AI)
AI/LLM Engineer (Model Fine-Tuning & Conversational AI)

Salve.Inno Consulting • Amsterdam

Remote
EUR 70,000 - 110,000
Fully remote (Europe)
B2B contractor agreement
20 paid leave days per year
+6
Senior Speech ML Engineer: TTS, Voice Cloning & AI
Senior Speech ML Engineer: TTS, Voice Cloning & AI

Cantina Labs • Netherlands

Remote
EUR 90,000 - 150,000
Competitive salary
Company equity
Medical, dental, vision insurance
+5
ML Engineer, Speech & Voice Conversion (End-to-End)
ML Engineer, Speech & Voice Conversion (End-to-End)

Cantina Labs • Netherlands

Remote
EUR 120,000 - 180,000
Equity
Medical/dental/vision insurance
42 days of PTO (paid time off)
+5
Principal ML Researcher
Principal ML Researcher

Toloka • Netherlands

Hybrid
EUR 150,000 - 220,000
Base salary + bonus
ESOP
Remote or hybrid model
+3
ML Researcher
ML Researcher

Desert Ant Labs • Amsterdam

Remote
EUR 70,000 - 120,000
AI/LLM Engineer (Model Fine-Tuning & Conversational AI)
AI/LLM Engineer (Model Fine-Tuning & Conversational AI)

Salve.Lab • Netherlands

Remote
EUR 90,000 - 140,000
Fully remote in Europe
B2B contractor arrangement
20 days paid leave
+3
Senior Machine Learning Engineer, Evals
Senior Machine Learning Engineer, Evals

Prosus • Amsterdam

Hybrid
EUR 120,000 - 180,000
Relocation support
Amsterdam office base
Remote work arrangements
+2