Arabic Localization Context Reviewer – Rubric & Quality

AuraOne, Inc.; United States; Remote; USD 28,000 - 55,000; Full time

Job summary

AuraOne, Inc. is seeking an Arabic Localization Cultural Context Reviewer for a remote contractor role. You will assess Arabic evaluation model outputs against a versioned rubric, assign severity tags, and provide structured feedback to support model retraining.

Strong attention to detail and experience with rubric-based evaluation are essential. Responsibilities include comparing paired outputs, labeling edge cases, and calibrating against gold-standard examples weekly.

Qualifications

• Experience evaluating Arabic generalist evaluation or related content.
• Ability to apply multi-page rubrics consistently across large batches.
• Clear written reasoning naming the issue and rubric clause.
• Attention to detail and ability to flag prompt problems.

Responsibilities

• Evaluate Arabic evaluation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger one with a rationale.
• Label hallucinations, policy violations, and unsafe content with structured tags.
• Document recurring failure modes and route prompts for rubric updates.

Skills

Model evaluation

Rubric annotation

Severity tagging

Inter-rater calibration

Arabic generalist evaluation

Localization review

Cultural context

Language evaluation

Arabic language

Localization

Job description

AuraOne, Inc. is seeking an Arabic Localization Cultural Context Reviewer for a remote contractor role. You will assess Arabic evaluation model outputs against a versioned rubric, assign severity tags, and provide structured feedback to support model retraining.

Strong attention to detail and experience with rubric-based evaluation are essential. Responsibilities include comparing paired outputs, labeling edge cases, and calibrating against gold-standard examples weekly.