Arabic Language Evaluation Specialist — Remote

AI Trainer Jobs; United States; Remote; USD 34,000 - 69,000; Part time

Job summary

AuraOne is seeking an Arabic Language Expert for a remote evaluation track. You will review Arabic language outputs, compare paired responses, and assign severity tags according to a formal rubric. This contractor role supports retraining and auditing of model outputs.

Responsibilities include labeling edge cases, writing structured feedback, and calibrating scores with gold-standard examples on a regular cadence. Remote, US-eligible, contractor arrangement.

Qualifications

• Experience evaluating language models or similar outputs.
• Ability to apply multi-page rubrics consistently.
• Clear written reasoning that names the issue and rubric clause.

Responsibilities

• Evaluate Arabic outputs against a versioned rubric and assign severity tags.
• Compare paired responses and provide rationale for the stronger answer.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Flag ambiguous prompts and request rubric updates as needed.
• Calibrate reviewer scores against gold-standard examples weekly.

Skills

Linguistics background

Careful reviewer

Arabic language proficiency

Annotation and feedback

Inter-rater calibration

Education

Background in linguistics or content moderation

Tools

Rubric-based annotation

Job description

AuraOne is seeking an Arabic Language Expert for a remote evaluation track. You will review Arabic language outputs, compare paired responses, and assign severity tags according to a formal rubric. This contractor role supports retraining and auditing of model outputs.

Responsibilities include labeling edge cases, writing structured feedback, and calibrating scores with gold-standard examples on a regular cadence. Remote, US-eligible, contractor arrangement.