Remote Audio Evaluation Engineer (Contract)

Agency; United States; Remote; USD 15,000 - 42,000; Full time

Job summary

Agency is conducting a project to assess and benchmark advanced agentic audio models for customer support scenarios. You will create role-play evaluation sessions across travel, financial services, telecom, and tech support domains.

You will help develop diverse evaluation datasets, measure model performance with qualitative and quantitative metrics, and ensure outputs reflect clear, efficient, natural conversations for real customers.

Qualifications

• Basic programming literacy including understanding JSON structures.
• Familiarity with functions and methods to reason about structured data.
• Technical communication clarity when handling support-style problem solving.

Responsibilities

• Create and execute role-play evaluation scenarios across multiple domains (flight bookings, travel, financial services, telecommunications, technical support).
• Contribute to development of diverse datasets and evaluate model performance using qualitative and quantitative metrics.

Skills

JSON structures

Structured data reasoning

Verbal communication

English proficiency

Tools

Audio recording software

Job description

Agency is conducting a project to assess and benchmark advanced agentic audio models for customer support scenarios. You will create role-play evaluation sessions across travel, financial services, telecom, and tech support domains.

You will help develop diverse evaluation datasets, measure model performance with qualitative and quantitative metrics, and ensure outputs reflect clear, efficient, natural conversations for real customers.