Remote German Audio Evaluation Specialist

Embedded Shishya; United States; Remote; USD 8,300 - 90,000; Part time

Job summary

Embedded Shishya is seeking independent Audio Evaluation Specialists for an AI benchmark project that simulates real-world customer service across travel, finance and telecom domains. You will create role-play scenarios and audit model outputs to generate clean evaluation data.

Requirements include proven customer support or conversational AI experience, fluent English, and basic JSON literacy. A microphone is required for high-quality audio input.

Qualifications

• Demonstrable professional expertise in complex customer support, technical troubleshooting, or conversational AI evaluation.
• Native or bilingual proficiency in English, with fluency in reading, listening, writing and speaking.
• Basic computer programming literacy, specifically understanding JSON structures, functions, and simple logic.
• Meticulous, detail-oriented approach to structured prompts, evaluation rubrics, and technical guidelines.
• Required equipment: access to a high-quality microphone for voice evaluations.

Responsibilities

• Role-Play Scenario Execution: Creating and executing complex, role-play-based evaluation scenarios that simulate realistic customer service interactions across travel, finance, and technical support domains.
• Model Performance Auditing: Evaluating AI model performance across standardized qualitative and quantitative metrics, focusing on task completion accuracy, conversational naturalness, and audio comprehension.
• Technical Metric Evaluation: Assessing the model's basic programming literacy, including understanding of JSON structures, functions, methods, and structured data reasoning.
• Representative Dataset Generation: Contributing to high-quality audio datasets reflecting real customer expectations for clarity and efficiency.

Skills

Customer support

Analytical thinking

Verbal communication

JSON literacy

Tools

JSON

Job description

Embedded Shishya is seeking independent Audio Evaluation Specialists for an AI benchmark project that simulates real-world customer service across travel, finance and telecom domains. You will create role-play scenarios and audit model outputs to generate clean evaluation data.

Requirements include proven customer support or conversational AI experience, fluent English, and basic JSON literacy. A microphone is required for high-quality audio input.