AI/LLM Speech & Audio AI Evaluation Specialist (Int. Voice)

Mindtel; Dadri; On-site; INR 900,000 - 1,300,000; Full time

Benefits offered by this job

On-site innovation lab in India

Flexible hours

Job summary

Mindtel is seeking an interpreter for an on-site role in India to evaluate and benchmark LLM-driven speech outputs across fluency, naturalness, and contextual accuracy. The candidate will design A/B tests, annotate datasets, and collaborate with ML engineers to inform model fine-tuning, with a focus on multilingual voice applications for fintech, healthcare, and telco sectors.

The role requires strong Python skills, experience with speech-to-text and text-to-speech systems, and familiarity with

Qualifications

• Proficiency in Python and ASR/TTS pipelines.
• Experience evaluating LLM-driven speech outputs using MOS/WER/CER.
• Experience annotating and labeling speech datasets for model training.
• Familiarity with multilingual speech and voice UX.

Responsibilities

• Evaluate and benchmark LLM-driven speech outputs across fluency, naturalness, emotional tone, and contextual accuracy using standardized metrics and custom rubrics.
• Design and run A/B tests for voice models across accents, registers, and cultural contexts to improve perceived quality and user trust.
• Annotate, transcribe, and label voice datasets for training and evaluation - focusing on turn-taking, intent alignment, and paralinguistic features.
• Collaborate with ML engineers to translate evaluation insights into fine-tuning signals and data augmentation strategies.
• Document voice quality trends, failure modes, and improvement pathways in structured reports for cross-functional review.
• Stay current with research in speech AI, prosody, and LLM evaluation - applying new methods like pairwise preference ranking or automated metrics (e.g., MOS, WER, CER).

Skills

Python

Speech-to-Text

Text-to-Speech

Speech Evaluation Metrics

LLM Evaluation

Audio Annotation Tools

Conversational AI

Multilingual Speech

Prosody Analysis

Paralinguistic Features

Voice UX Design

Job description

About the Opportunity

A fast-growing AI innovation lab focused on next-gen voice and speech intelligence, we build and evaluate cutting-edge Large Language Models (LLMs) for real-world conversational AI applications. Our work spans multilingual voice agents, prosody modeling, and context-aware synthesis - serving global clients across fintech, healthcare, and telco sectors. We move fast, validate rigorously, and ship voice experiences that sound human, behave intelligently, and scale reliably.

Role & Responsibilities

• Evaluate and benchmark LLM-driven speech outputs across fluency, naturalness, emotional tone, and contextual accuracy using standardized metrics and custom rubrics.
• Design and run A/B tests for voice models across accents, registers, and cultural contexts to improve perceived quality and user trust.
• Annotate, transcribe, and label voice datasets for training and evaluation - focusing on turn-taking, intent alignment, and paralinguistic features.
• Collaborate with ML engineers to translate evaluation insights into fine-tuning signals and data augmentation strategies.
• Document voice quality trends, failure modes, and improvement pathways in structured reports for cross-functional review.
• Stay current with research in speech AI, prosody, and LLM evaluation - applying new methods like pairwise preference ranking or automated metrics (e.g., MOS, WER, CER).

Skills & Qualifications

• Must-Have
• Python
• Speech-to-Text
• Text-to-Speech
• Speech Evaluation Metrics
• LLM Evaluation
• Audio Annotation Tools
• Conversational AI
• Multilingual Speech
• Preferred
• Prosody Analysis
• Paralinguistic Features
• Voice UX Design

Benefits & Culture Highlights

• Work on bleeding-edge voice AI projects with global impact.
• Collaborative, research-driven culture with access to top-tier LLM tooling.
• On-site innovation lab environment in India with flexible hours and peer learning.