Senior AI Voice Quality & Evaluation Analyst

Spotify; Greater London; Hybrid; GBP 80,000 - 120,000; Full time

Job summary

Spotify London-based Senior Product Quality Analyst leads the evaluation of AI voice models our Voice Production and Speak teams use. You’ll define what quality means, design rubrics, and run qualitative and MOS-style assessments to guide product decisions across languages and regions.

You’ll partner with ML, product, and vendor teams to ensure we deliver consistent, high-quality voice experiences at scale while balancing multiple priorities and markets.

Qualifications

• Experience in product quality, evaluation or related field using human judgment to assess performance.
• Ability to define evaluation criteria and create repeatable rubrics.
• Skilled in designing and running qualitative and MOS-style evaluations.

Responsibilities

• Own the quality bar for Spotify AI voice models across products, audiences and use cases.
• Design evaluation rubrics and an approach measuring naturalness, tone, pacing and pronunciation.
• Plan and execute evaluation campaigns; translate findings into actionable recommendations.
• Establish ongoing evaluation cadence as technology evolves.
• Benchmark against third-party models to identify improvement opportunities.
• Determine quality trade-offs for new languages and regions.
• Collaborate with voice experts, ML teams and vendors to maintain quality.

Skills

Product quality

Evaluation

Data curation

Qualitative evaluation

MOS-style evaluation

Voice quality

Autonomous working

Job description

Spotify London-based Senior Product Quality Analyst leads the evaluation of AI voice models our Voice Production and Speak teams use. You’ll define what quality means, design rubrics, and run qualitative and MOS-style assessments to guide product decisions across languages and regions.

You’ll partner with ML, product, and vendor teams to ensure we deliver consistent, high-quality voice experiences at scale while balancing multiple priorities and markets.