Remote Agentic Audio Evaluation Specialist

remote work; United States; Remote; USD 15,000 - 42,000; Part time

Job summary

Meridial is evaluating and benchmarking advanced agentic audio models for real-world customer support use cases. This role focuses on creating evaluation scenarios, collecting datasets, and assessing model performance against defined metrics.

You will design and execute role-play based evaluation scenarios across travel, finance, telecom and technical support domains, contributing to diverse datasets and evaluating performance with both qualitative and quantitative metrics.

Qualifications

• Strong verbal communication in professional English.
• Understanding of JSON structures and basic programming concepts.
• Ability to reason about structured data and simple logic.
• Comfort with evaluating using rubrics and technical guidelines.
• Experience with audio evaluations or customer support scenarios is a plus.

Responsibilities

• Design and execute role-play based evaluation scenarios across travel, finance, telecom and technical support domains.
• Contribute to diverse and representative datasets for evaluating conversational audio agents.
• Evaluate performance using qualitative and quantitative metrics such as task completion, naturalness, and audio quality.
• Analyze structured data and JSON related outputs to support evaluation outcomes.
• Document evaluation results and provide clarity on instruction adherence and contextual understanding.

Skills

Verbal communication

English proficiency

JSON basics

Structured data reasoning

Rubrics and guidelines

Job description

Meridial is evaluating and benchmarking advanced agentic audio models for real-world customer support use cases. This role focuses on creating evaluation scenarios, collecting datasets, and assessing model performance against defined metrics.

You will design and execute role-play based evaluation scenarios across travel, finance, telecom and technical support domains, contributing to diverse datasets and evaluating performance with both qualitative and quantitative metrics.