Portuguese AI Audio Evaluator & Dialogue Specialist

Agency; United States; Remote; USD 80,000 - 110,000; Full time

Job summary

Agency in the United States conducts advanced evaluations of AI customer-support agents through role-play scenarios and detailed metrics. The role demands Portuguese fluency and expertise in complex support, troubleshooting, or AI evaluation, plus basic programming literacy and reliable audio capture.

You will help shape high-quality audio datasets and assess model behavior on both qualitative and quantitative scales, ensuring rigorous guidelines are followed.

Qualifications

• Native or bilingual proficiency in Portuguese.
• Professional expertise in complex customer support, technical troubleshooting, or conversational AI evaluation.
• Basic computer programming literacy (JSON, functions, methods, and simple logic).
• Access to a high-quality microphone for clean audio input.

Responsibilities

• Create and execute complex, role-play-based evaluation scenarios simulating realistic customer service interactions.
• Evaluate AI model performance across qualitative and quantitative metrics, focusing on task completion and naturalness.
• Assess model computer programming literacy, including JSON structures and reasoning about structured data.
• Contribute to development of diverse, high-quality audio datasets reflecting real customer expectations.

Skills

Portuguese proficiency

AI evaluation expertise

Technical troubleshooting

JSON basics

Audio data collection

Tools

JSON

Job description

Agency in the United States conducts advanced evaluations of AI customer-support agents through role-play scenarios and detailed metrics. The role demands Portuguese fluency and expertise in complex support, troubleshooting, or AI evaluation, plus basic programming literacy and reliable audio capture.

You will help shape high-quality audio datasets and assess model behavior on both qualitative and quantitative scales, ensuring rigorous guidelines are followed.