Remote Russian Bilingual Evaluator & Rubric Feedback

AuraOne; United States; On-site; USD 28,000 - 55,000; Part time

Job summary

AuraOne seeks a remote contractor to evaluate Russian generalist outputs, applying a fixed rubric and labeling severity. You will compare paired responses, provide justifications, and tag unsafe content. This role emphasizes clear written reasoning and consistency across long batches.

Ideal candidates have prior evaluation experience and can commit at least 10 hours per week, with US remote eligibility. Compensation is hourly and determined after the interview process.

Qualifications

• Prior evaluation, annotation, or human-rater experience on russian generalist evaluation or adjacent content.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate russian generalist evaluation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and provide a rationale for the stronger answer.
• Label unsafe content and policy violations with structured tags.
• Capture ambiguous prompts and route them for rubric updates.
• Calibrate reviewer scores against gold-standard examples weekly.
• Document recurring failure modes to guide training improvements.

Skills

Russian evaluation

Annotation

Rubric-based annotation

Inter-rater calibration

Quality assurance

Job description

AuraOne seeks a remote contractor to evaluate Russian generalist outputs, applying a fixed rubric and labeling severity. You will compare paired responses, provide justifications, and tag unsafe content. This role emphasizes clear written reasoning and consistency across long batches.

Ideal candidates have prior evaluation experience and can commit at least 10 hours per week, with US remote eligibility. Compensation is hourly and determined after the interview process.