Remote Italian Evaluator & Rubric Feedback Specialist

AuraOne; United States; On-site; USD 19,000 - 25,000; Full time

Job summary

AuraOne is seeking a Bilingual Writer - Italian (Italy) for a remote evaluation track. You will review italian generalist evaluation prompts and responses, compare paired outputs, and provide structured feedback to retrain the model.

Responsibilities include labeling edge cases, annotating content with rubric tags, and maintaining reviewer-quality scores against gold-standard examples. Remote-US eligible contractor role.

Qualifications

• Experience evaluating Italian generalist content or similar tasks.
• Ability to apply multi-page rubrics consistently across batches.
• Clear written reasoning that cites rubric clauses.

Responsibilities

• Evaluate Italian outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger with rationale.
• Label hallucinations and unsafe content with structured tags.
• Document recurring failures and propose rubric clarifications.
• Calibrate scores against gold-standard examples weekly.
• Route ambiguous prompts to the program team for updates.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Italian generalist evaluation

Job description

AuraOne is seeking a Bilingual Writer - Italian (Italy) for a remote evaluation track. You will review italian generalist evaluation prompts and responses, compare paired outputs, and provide structured feedback to retrain the model.

Responsibilities include labeling edge cases, annotating content with rubric tags, and maintaining reviewer-quality scores against gold-standard examples. Remote-US eligible contractor role.