German Audio Evaluations Specialist

Agency; United States; Remote; USD 60,000 - 110,000; Full time

Job summary

Agency is seeking a contributor to test and optimize advanced agentic audio models in real-world customer support scenarios across sectors such as flight bookings, financial services, and telecommunications. The role focuses on creating realistic evaluation prompts, scoring model performance, and ensuring repeatable, high-quality audio data collection.

You will work with complex prompts, analyze qualitative and quantitative results, and ensure precise handling of structured prompts and

Qualifications

• Native or bilingual proficiency in German.
• Experience in complex customer support or AI evaluation.
• Basic JSON programming literacy (JSON, functions, methods).
• Access to a high-quality microphone for audio input.
• Meticulous, detail-oriented approach to structured prompts and technical guidelines.

Responsibilities

• Create and execute complex, role-play-based evaluation scenarios simulating realistic customer service interactions.
• Evaluate AI model performance across qualitative and quantitative metrics, focusing on task completion and naturalness.
• Assess model programming literacy, including JSON structures and reasoning about structured data.
• Contribute to the development of diverse, high-quality audio datasets.

Skills

German language

Customer support experience

AI evaluation

Tools

JSON

Job description

Responsibilities

• Create and execute complex, role-play-based evaluation scenarios simulating realistic customer service interactions
• Evaluate AI model performance across qualitative and quantitative metrics, focusing on task completion and naturalness
• Assess model programming literacy, including JSON structures and reasoning about structured data
• Contribute to the development of diverse, high-quality audio datasets

Requirements

• Native or bilingual proficiency in German
• Professional expertise in complex customer support, technical troubleshooting, or conversational AI evaluation
• Basic computer programming literacy (JSON, functions, methods, and simple logic)
• Access to a high-quality microphone for clean audio input
• Meticulous, detail-oriented approach to structured prompts and technical guidelines

About the Company

This project focuses on assessing advanced agentic audio models to optimize performance in real-world customer support scenarios like flight bookings, financial services, and telecommunications.