Mandarin Audio Evaluation Specialist (Remote)

AI Trainer Jobs; United States; Remote; USD 41,000 - 74,000; Part time

Benefits offered by this job

Remote work

Contractor role

Job summary

AuraOne is seeking a Music Audio Expert - Mandarin Chinese for a remote evaluation track. You will review Mandarin prompts and responses against a quality rubric, compare outputs, and provide structured feedback to retrain models.

This contractor role focuses on labeling edge cases, calibration, and ensuring consistent quality across batches. Responsibilities include evaluating model outputs, tagging issues, and maintaining calibration with weekly gold-standard comparisons.

Qualifications

• Experience evaluating Mandarin generalist outputs using rubrics.
• Ability to apply rubrics consistently across long batches.
• Clear written reasoning naming the issue and rubric clause.

Responsibilities

• Evaluate Mandarin generalist evaluation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples weekly.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Mandarin generalist evaluation

Job description

AuraOne is seeking a Music Audio Expert - Mandarin Chinese for a remote evaluation track. You will review Mandarin prompts and responses against a quality rubric, compare outputs, and provide structured feedback to retrain models.

This contractor role focuses on labeling edge cases, calibration, and ensuring consistent quality across batches. Responsibilities include evaluating model outputs, tagging issues, and maintaining calibration with weekly gold-standard comparisons.