Agentic Audio Evaluation Specialist

Agency; United States; Remote; USD 34,000 - 55,000; Full time

Job summary

Agency is seeking to join a project focused on assessing and benchmarking advanced agentic audio models used in real-world customer support scenarios. You will design and execute role-play–based evaluations across travel, finance, and telecommunications contexts to simulate authentic customer interactions.

Contribute to datasets, measure model performance with qualitative and quantitative metrics, and help ensure evaluations reflect customer expectations for clarity, efficiency, and natural flow

Qualifications

• Strong verbal communication skills in a simulated customer support context.
• English proficiency across reading, listening, writing, and speaking.
• Access to a high-quality microphone for clean audio input.
• Basic computer programming literacy, including JSON structures and familiarity with functions and methods.
• Ability to reason about structured data and simple logic.
• Device capable of running audio recording software and handling large technical documentation.

Responsibilities

• Create and execute role-play–based evaluation scenarios simulating realistic customer service interactions in travel, finance, and telecommunications
• Contribute to the development of diverse and representative datasets for conversational audio agents
• Evaluate model performance using qualitative and quantitative metrics
• Ensure evaluations reflect real customer expectations for clarity, efficiency, and natural conversational flow

Skills

Verbal communication

English proficiency

Programming literacy

Tools

JSON structures

Audio recording software

Job description

Agency is seeking to join a project focused on assessing and benchmarking advanced agentic audio models used in real-world customer support scenarios. You will design and execute role-play–based evaluations across travel, finance, and telecommunications contexts to simulate authentic customer interactions.

Contribute to datasets, measure model performance with qualitative and quantitative metrics, and help ensure evaluations reflect customer expectations for clarity, efficiency, and natural flow