German AI Audio Evaluation Specialist

Agency; United States; Remote; USD 60.000 - 110.000; Vollzeit

Zusammenfassung

Agency is seeking a contributor to test and optimize advanced agentic audio models in real-world customer support scenarios across sectors such as flight bookings, financial services, and telecommunications. The role focuses on creating realistic evaluation prompts, scoring model performance, and ensuring repeatable, high-quality audio data collection.

You will work with complex prompts, analyze qualitative and quantitative results, and ensure precise handling of structured prompts and

Qualifikationen

• Native or bilingual proficiency in German.
• Experience in complex customer support or AI evaluation.
• Basic JSON programming literacy (JSON, functions, methods).
• Access to a high-quality microphone for audio input.
• Meticulous, detail-oriented approach to structured prompts and technical guidelines.

Aufgaben

• Create and execute complex, role-play-based evaluation scenarios simulating realistic customer service interactions.
• Evaluate AI model performance across qualitative and quantitative metrics, focusing on task completion and naturalness.
• Assess model programming literacy, including JSON structures and reasoning about structured data.
• Contribute to the development of diverse, high-quality audio datasets.

Kenntnisse

German language

Customer support experience

AI evaluation

Tools

JSON

Jobbeschreibung

Agency is seeking a contributor to test and optimize advanced agentic audio models in real-world customer support scenarios across sectors such as flight bookings, financial services, and telecommunications. The role focuses on creating realistic evaluation prompts, scoring model performance, and ensuring repeatable, high-quality audio data collection.

You will work with complex prompts, analyze qualitative and quantitative results, and ensure precise handling of structured prompts and