Welsh Audio Specialist

Agency; United States; A distancia; USD 60.000 - 90.000; Jornada completa

Descripción de la vacante

Agency seeks an evaluator to design and run advanced role-play scenarios that mimic real customer service interactions. You will assess AI models on task completion, natural language quality, and basic JSON reasoning.

The role requires Welsh proficiency and prior experience in complex support or AI evaluation, with access to reliable audio input equipment. You will help build diverse audio datasets for real-world scenarios.

Formación

• Native or bilingual proficiency in Welsh.
• Experience in complex customer support or AI evaluation.
• Basic JSON and programming literacy.
• Access to a high-quality microphone for clear audio input.
• Meticulous, detail-oriented approach to structured prompts and rubrics.

Responsabilidades

• Create and execute complex, role-play-based evaluation scenarios simulating realistic customer service interactions.
• Evaluate AI model performance across qualitative and quantitative metrics, focusing on task completion and conversational naturalness.
• Assess model's basic programming literacy, including JSON structures and reasoning about structured data.
• Contribute to the development of diverse, high-quality audio datasets reflecting real customer expectations.

Conocimientos

Customer support

AI evaluation

Data analysis

Attention to detail

Herramientas

JSON literacy

Audio tools

Descripción del empleo

Responsibilities

• Create and execute complex, role-play-based evaluation scenarios simulating realistic customer service interactions
• Evaluate AI model performance across qualitative and quantitative metrics, focusing on task completion and conversational naturalness
• Assess model's basic computer programming literacy, including JSON structures and reasoning about structured data
• Contribute to the development of diverse, high-quality audio datasets reflecting real customer expectations

Requirements

• Native or bilingual proficiency in Welsh
• Professional expertise in complex customer support, technical troubleshooting, or conversational AI evaluation
• Basic computer programming literacy (JSON, functions, methods, and simple logic)
• Access to a high-quality microphone for clean audio input
• Meticulous, detail-oriented approach to structured prompts and evaluation rubrics

About the Company

This project focuses on assessing advanced agentic audio models for real-world customer support scenarios like flight bookings, financial services, and telecommunications.