Global Multilingual Audio Evaluator - Remote Contractor

Anyone AI Inc.; Roma; Remote; EUR 19,000 - 49,000; Part time

Job summary

Anyone AI Inc. is seeking multilingual evaluators to assess conversational AI across languages. You will participate in live roleplay sessions, capture recordings and logs, label transcripts, and score interactions using structured rubrics.

Candidates should have C1/C2 level language proficiency, strong JSON skills, and meticulous attention to detail. This project-based, remote role involves working with diverse languages and providing clear written explanations of scores.

Qualifications

• C1/C2 spoken fluency in at least one target language from the list.
• Proven ability to validate and troubleshoot JSON data.
• Strong attention to detail and ability to follow evaluation guidelines.

Responsibilities

• Prepare and confirm required tools and access before each live session.
• Conduct live, real-time roleplay scenarios with conversational AI agents.
• Capture recordings, session IDs, and tool-call or console logs.
• Label conversation transcripts according to project guidelines.
• Score interactions against structured evaluation rubrics.
• Provide clear written explanations supporting scores.
• Read, validate, and troubleshoot JSON, identifying issues.

Skills

Language proficiency

JSON validation

Attention to detail

Clear written communication

Tools

JSON tooling

Job description

Anyone AI Inc. is seeking multilingual evaluators to assess conversational AI across languages. You will participate in live roleplay sessions, capture recordings and logs, label transcripts, and score interactions using structured rubrics.

Candidates should have C1/C2 level language proficiency, strong JSON skills, and meticulous attention to detail. This project-based, remote role involves working with diverse languages and providing clear written explanations of scores.