Remote Portuguese Audio AI Evaluator & Trainer

Agency; United States; Remote; USD 8,300 - 90,000; Full time

Job summary

We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project. Contractors will design evaluation frameworks, execute role-play scenarios, and audit AI outputs to generate clean datasets for model improvement.

Required expertise includes deep customer support knowledge, bilingual fluency, and basic JSON literacy. Engagement is freelance/independent contractor with remote/work-from-home setup; equipment like a high-quality microphone is expected.

Qualifications

• Deep expertise in complex customer support, troubleshooting, or conversational AI evaluation.
• Native or bilingual proficiency in the target language with strong reading, listening, writing and speaking skills.
• Basic programming literacy with JSON structures, functions, and simple logic.

Responsibilities

• Design and execute complex role-play evaluation scenarios across travel, finance, and technical support domains.
• Audit model outputs using standardized qualitative and quantitative metrics for task completion, naturalness, and audio comprehension.
• Evaluate model literacy in JSON, functions, methods, and handling of structured data in a support context.
• Contribute to diverse, high-quality audio datasets reflecting real customer expectations for clarity and efficiency.

Skills

Audio Evaluation

JSON Basics

Quality Evaluation

Bilingual Proficiency

Job description

We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project. Contractors will design evaluation frameworks, execute role-play scenarios, and audit AI outputs to generate clean datasets for model improvement.

Required expertise includes deep customer support knowledge, bilingual fluency, and basic JSON literacy. Engagement is freelance/independent contractor with remote/work-from-home setup; equipment like a high-quality microphone is expected.