Remote Urdu–English Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 21,000 - 28,000; Full time

Job summary

AuraOne is seeking an Urdu-English Language Evaluation Specialist for a remote evaluation track. Review outputs, compare paired responses, label edge cases, and provide structured feedback for retraining the model.

The role involves AI data review, tagging of issues, and calibration against gold standards. This is a contractor position with US-eligible remote work and flexible scheduling.

Qualifications

• Prior evaluation, annotation, or human-rater experience on Urdu generalist evaluation.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate Urdu generalist evaluation model outputs against a versioned rubric and assign severity tags for Urdu-English Language Evaluation Specialist assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.

Skills

Urdu evaluation

Annotation

Rationale writing

Inter-rater calibration

Linguistics background

Job description

AuraOne is seeking an Urdu-English Language Evaluation Specialist for a remote evaluation track. Review outputs, compare paired responses, label edge cases, and provide structured feedback for retraining the model.

The role involves AI data review, tagging of issues, and calibration against gold standards. This is a contractor position with US-eligible remote work and flexible scheduling.