Remote Bilingual Model Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 68,000 - 135,000; Part time

Job summary

AuraOne is seeking evaluators to remotely review bilingual generalists prompts and responses, scoring them against a versioned rubric. You will compare paired outputs, label edge cases, and draft structured feedback for retraining the modeling team.

Responsibilities include identifying hallucinations, instruction-following failures, and unsafe content with precise tags, while maintaining calibration against gold standards and ensuring consistent rubric application.

Qualifications

• Prior evaluation, annotation, or human-rater experience on bilingual generalists evaluation or adjacent content.
• Clear written reasoning naming the issue and the rubric clause applied.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with rationale.
• Label hallucinations and policy violations with structured tags.
• Capture ambiguous prompts and coordinate rubric updates with the team.
• Calibrate reviewer quality against gold-standard examples weekly.

Skills

Attention to detail

Written reasoning

Async availability

Bilingual proficiency

Education

Linguistics background

Tools

Annotation tools

Job description

AuraOne is seeking evaluators to remotely review bilingual generalists prompts and responses, scoring them against a versioned rubric. You will compare paired outputs, label edge cases, and draft structured feedback for retraining the modeling team.

Responsibilities include identifying hallucinations, instruction-following failures, and unsafe content with precise tags, while maintaining calibration against gold standards and ensuring consistent rubric application.