Remote Russian Audio Evaluator Expert - $50/hr Contractor

AI Trainer Jobs; United States; Remote; USD 66,000 - 72,000; Full time

Job summary

AI Trainer Jobs is seeking a Russian Audio Generalist Evaluator Expert (San Francisco Bay Area) for a remote evaluation track to review prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and provide structured feedback to retrain the model.

Responsibilities include evaluating outputs against a versioned rubric, selecting the stronger answer with rationale, and tagging hallucinations or safety concerns.

Qualifications

• Prior evaluation, annotation, or human-rater experience on Russian generalist evaluation.
• Ability to provide clear written reasoning with rubric references.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate model outputs against rubric and tag severity.
• Compare paired responses and justify the stronger answer.
• Label hallucinations, safety content and instruction-following failures.
• Capture ambiguous prompts for rubric updates.

Skills

Rubric annotation

Severity tagging

Inter-rater calibration

Russian evaluation

Job description

AI Trainer Jobs is seeking a Russian Audio Generalist Evaluator Expert (San Francisco Bay Area) for a remote evaluation track to review prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and provide structured feedback to retrain the model.

Responsibilities include evaluating outputs against a versioned rubric, selecting the stronger answer with rationale, and tagging hallucinations or safety concerns.