Remote Italian Audio Evaluation Specialist

Agency; United States; Remote; USD 8,300 - 90,000; Full time

Job summary

Agency is seeking independent Audio Evaluation Specialists for an AI benchmark project focused on agentic audio models. You will design evaluation tasks, run realistic customer-service role-plays, and provide high-quality data.

Requirements include native or bilingual language skills, strong communication, and basic JSON literacy. This is a remote freelance engagement with pay between $6 and $65 per hour, determined by experience and location.

Qualifications

• Must have professional experience in complex customer support testing or conversational AI evaluation.
• Native or bilingual proficiency in English (reading, listening, writing and speaking).
• Basic programming literacy with JSON structures and simple logic.
• Meticulous and precise when handling structured prompts and rubrics.

Responsibilities

• Role-Play Scenario Execution: create and perform complex evaluation scenarios across travel, finance, and technical support.
• Model Performance Auditing: evaluate AI outputs using standardized qualitative and quantitative metrics.
• Technical Metric Evaluation: assess JSON understanding within a support context.
• Representative Dataset Generation: contribute to high-quality audio datasets reflecting real customer expectations.

Skills

Customer support

Bilingual English

JSON basics

Detail-oriented

Job description

Agency is seeking independent Audio Evaluation Specialists for an AI benchmark project focused on agentic audio models. You will design evaluation tasks, run realistic customer-service role-plays, and provide high-quality data.

Requirements include native or bilingual language skills, strong communication, and basic JSON literacy. This is a remote freelance engagement with pay between $6 and $65 per hour, determined by experience and location.