Remote Welsh Audio Evaluation Specialist

Agency; United States; Remote; USD 8,300 - 90,000; Full time

Benefits offered by this job

Remote work

Freelance/Independent contractor

Job summary

Agency is seeking independent Audio Evaluation Specialists for an AI benchmark project. You will design complex evaluation scenarios, simulate realistic customer service conversations, and audit AI outputs to produce clean, reliable datasets for model improvement.

We require deep expertise in customer support or conversational AI evaluation, with strong language skills and basic programming literacy in JSON. This is a freelance, remote engagement using high-quality microphones.

Qualifications

• Demonstrated experience in complex customer support or technical troubleshooting.
• Experience evaluating conversational AI, with attention to accuracy and naturalness.

Responsibilities

• Design and run role-play based evaluation scenarios across travel, finance, and technical support domains.
• Audit AI model outputs using structured rubrics and qualitative/quantitative metrics.
• Generate high-quality audio datasets reflecting realistic customer expectations.

Skills

Customer support

Technical troubleshooting

Conversational AI evaluation

Tools

JSON structures

Job description

Agency is seeking independent Audio Evaluation Specialists for an AI benchmark project. You will design complex evaluation scenarios, simulate realistic customer service conversations, and audit AI outputs to produce clean, reliable datasets for model improvement.

We require deep expertise in customer support or conversational AI evaluation, with strong language skills and basic programming literacy in JSON. This is a freelance, remote engagement using high-quality microphones.