Multilingual AI Quality Specialist

Creandum Advisor LLP

Stockholms kommun

Ibrido

SEK 776.397 - 1.358.695

Tempo pieno

14 giorni+

Una candidatura completa in un minuto

Curriculum e lettera di presentazione personalizzati, pronti da inviare.

Supera i filtri ATS

Descrizione del lavoro

Spotify is seeking a professional to advance multilingual AI data quality across Localization. You will define quality standards, design evaluations, and curate multilingual datasets to support scalable AI model development and evaluation.

Based in London or Stockholm, you will partner with Product, Engineering, Data Science, Research, and Localization to drive quality signals, use cases, and launch decisions, while embracing flexible work arrangements.

Competenze

  • Experience in multilingual quality evaluation, localization, data curation, annotation, or related fields.
  • Understanding language quality, cultural relevance, and user experience across languages and markets.
  • Familiar with structured evaluations using rubrics, audits, or annotation workflows.
  • Comfort using qualitative and quantitative data to identify trends and improve quality.
  • Knowledge of LLMs, AI evaluation, and human‑in‑the‑loop workflows is a plus.
  • Ability to work in cross‑functional environments.

Mansioni

  • Define quality frameworks, rubrics, thresholds for multilingual AI experiences.
  • Design and execute structured evaluations for AI-generated experiences.
  • Lead multilingual dataset curation, annotation, and ground-truth creation.
  • Analyze results, identify gaps, and provide actionable recommendations.
  • Support evaluator calibration and human‑AI agreement studies.
  • Collaborate with Product, Engineering, Data Science, Localization, vendors and markets.
  • Document best practices and help define quality standards.
  • Contribute to scalable evaluation capabilities for next-gen AI experiences.

Conoscenze

Quality evaluation
Localization
Data curation
Annotation
AI evaluation
LLMs awareness

Strumenti

SQL
Python
Dashboards
Annotation platforms

Descrizione del lavoro

The Platform team creates the technology that enables Spotify to learn quickly and scale easily, enabling rapid growth in our users and our business around the globe. Spanning many disciplines, we work to make the business work; creating the infrastructure, tooling, frameworks, and capabilities needed to welcome a billion customers.

The Multilingual AI Data Quality space is part of Spotify's Global Language Quality Program within Localization. Bringing together multilingual language quality evaluation and AI data quality, the team helps ensure our AI-powered experiences are accurate, culturally relevant, and trustworthy across languages and markets.

Working closely with Product, Engineering, Data Science, Research, Personalization, Localization, vendors, and market experts, the team develops evaluation methodologies, high-quality datasets, and quality signals that help improve AI experiences and inform product decisions at scale.

What You’ll Do:
  • Define quality frameworks, evaluation rubrics, thresholds, and methodologies for multilingual AI experiences.
  • Design and execute structured evaluations for AI-generated, AI-translated, AI-curated, and recommendation-driven experiences.
  • Lead multilingual dataset curation, annotation, enrichment, and ground-truth creation to support AI model development and evaluation.
  • Analyze evaluation results, identify quality gaps, and provide actionable recommendations to improve multilingual AI quality.
  • Support LLM-as-a-judge workflows, evaluator calibration, and human‑AI agreement studies.
  • Partner closely with Product, Engineering, Data Science, Research, Localization, vendors, and market experts to improve AI quality signals and inform launch decisions.
  • Document best practices and help define quality standards across languages, markets, and AI use cases.
  • Contribute to building scalable evaluation capabilities that support the next generation of AI‑powered experiences across Spotify.
Who You Are:
  • You have experience in multilingual quality evaluation, localization, data curation, annotation, AI evaluation, or related fields, including text‑to‑text and text‑to‑speech experiences.
  • You understand language quality, cultural relevance, content quality, and user experience across multiple languages and markets.
  • You have experience designing or conducting structured evaluations using quality rubrics, audits, annotation projects, or review methodologies.
  • You are comfortable using qualitative and quantitative data to identify trends, measure quality, and make recommendations.
  • You are familiar with large language models (LLMs), generative AI evaluation, human‑in‑the‑loop workflows, or LLM‑as‑a‑judge methodologies.
  • You enjoy working through ambiguity and turning complex quality challenges into practical evaluation strategies.
  • You communicate effectively and thrive in highly cross‑functional environments, collaborating with technical and non‑technical partners alike.
  • Experience with recommendation systems, personalization, search, ranking, machine translation, generative AI, dataset creation, annotation operations, evaluator calibration, prompt testing, model evaluation, SQL, Python, dashboards, or annotation platforms is a plus.
Where You’ll Be:
  • This role is based in London or Stockholm.
  • We offer you the flexibility to work where you work best! There will be some in person meetings, but still allows for flexibility to work from home.

Spotify is an equal opportunity employer. You are welcome at Spotify for who you are, no matter where you come from, what you look like, or what’s playing in your headphones. Our platform is for everyone, and so is our workplace. The more voices we have represented and amplified in our business, the more we will all thrive, contribute, and be forward‑thinking! So bring us your personal experience, your perspectives, and your background. It’s in our differences that we will find the power to keep revolutionizing the way the world listens.

At Spotify, we are passionate about inclusivity and making sure our entire recruitment process is accessible to everyone. We have ways to request reasonable accommodations during the interview process and help assist in what you need. If you need accommodations at any stage of the application or interview process, please let us know - we’re here to support you in any way we can.

Ottieni la revisione del curriculum gratis e riservata.

o trascina qui il file.

Similar jobs

Offerte di lavoro simili che vale la pena confrontare

Remote Multilingual AI Quality & Evaluation Lead
Remote Multilingual AI Quality & Evaluation Lead

Creandum Advisor LLP • Stockholms kommun

Ibrido
SEK 776.397 - 1.358.695
Senior Applied Research Scientist - Personalization
Senior Applied Research Scientist - Personalization

Spotify • Stockholms kommun

Ibrido
SEK 900.000 - 1.300.000
Flexible work options
Health insurance
Machine Learning Engineering Manager II
Machine Learning Engineering Manager II

Spotify • Stockholms kommun

Ibrido
SEK 1.200.000 - 1.900.000
Senior Machine Learning Engineer - Policy & Safety
Senior Machine Learning Engineer - Policy & Safety

Spotify • Stockholm

Ibrido
SEK 900.000 - 1.100.000
Backend Engineer - Data Platform
Backend Engineer - Data Platform

Spotify • Stockholms kommun

Ibrido
SEK 900.000 - 1.200.000
Staff Machine Learning Engineer - Content Intelligence
Staff Machine Learning Engineer - Content Intelligence

Spotify • Stockholms kommun

Ibrido
SEK 758.478 - 1.300.249
Senior Digital Learning Experience Designer, Learning & Inclusion
Senior Digital Learning Experience Designer, Learning & Inclusion

Spotify • Stockholms kommun

Ibrido
SEK 884.000 - 1.179.000
Senior Machine Learning Engineer - Messaging Platform
Senior Machine Learning Engineer - Messaging Platform

Spotify • Stockholms kommun

Ibrido
SEK 600.000 - 900.000
Senior Staff Machine Learning Engineer, Content Platform
Senior Staff Machine Learning Engineer, Content Platform

Spotify • Stockholms kommun

Ibrido
SEK 70.000 - 120.000
Staff Engineer - Content Intelligence Infrastructure
Staff Engineer - Content Intelligence Infrastructure

Spotify • Stockholms kommun

Ibrido
SEK 600.000 - 900.000