Italian Evaluation Specialist (Remote) – Rubric QA

AuraOne Human Data

United States

Remote

USD 45.000 - 65.000

Vollzeit

Vor 2 Tagen
Sei unter den ersten Bewerbenden

Erhalte eine Antwort von diesem Arbeitgeber

Dein Lebenslauf und dein Anschreiben, genau auf das zugeschnitten, was gesucht wird.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

AuraOne Human Data seeks an Italian Linguistic Expert for remote evaluation tasks. You will assess Italian outputs against a rubric, label issues, and provide structured feedback to retrain models.

You will compare paired responses, justify your choice, and tag content by severity to support quality improvements. Ideal candidates will show prior evaluation experience, strong attention to detail, and the ability to work asynchronously at least 10 hours weekly from a US-eligible remote setup.

Qualifikationen

  • Prior evaluation, annotation, or human-rater experience on italian generalist evaluation or adjacent content for Italian Linguistic Expert work.
  • Comfort applying multi-page rubrics consistently across long batches.
  • Clear written reasoning that names the issue and the rubric clause being applied.
  • Strong attention to detail and the ability to flag when a prompt itself is the problem.
  • Reliable async availability for at least 10 hours per week.

Aufgaben

  • Evaluate italian generalist evaluation model outputs against a versioned rubric and assign severity tags for Italian Linguistic Expert assignments.
  • Compare paired responses and pick the stronger answer with a written rationale.
  • Label hallucinations, instruction-following failures, and unsafe content with structured tags.
  • Capture ambiguous prompts and route them back to the program team for rubric updates.
  • Maintain reviewer-quality scores by calibrating against gold-standard examples each week.
  • Document recurring failure modes so the modeling team can target them in the next training run.

Kenntnisse

Model output evaluation
Rubric-based annotation
Severity tagging
Inter-rater calibration
Italian generalist evaluation

Jobbeschreibung

AuraOne Human Data seeks an Italian Linguistic Expert for remote evaluation tasks. You will assess Italian outputs against a rubric, label issues, and provide structured feedback to retrain models.

You will compare paired responses, justify your choice, and tag content by severity to support quality improvements. Ideal candidates will show prior evaluation experience, strong attention to detail, and the ability to work asynchronously at least 10 hours weekly from a US-eligible remote setup.

Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

Remote Italian Bilingual Evaluator and Feedback Specialist
Remote Italian Bilingual Evaluator and Feedback Specialist

Auraone • USA

Remote
USD 18.000 - 26.000
Remote work
Italian Linguistic Expert
Italian Linguistic Expert

AuraOne Human Data • USA

Remote
USD 45.000 - 65.000
Remote Italian Music & Lyrics Evaluation Specialist
Remote Italian Music & Lyrics Evaluation Specialist

Auraone • USA

Remote
USD 28.000 - 50.000
Remote work
Italian AI Evaluation & Annotation Specialist - Remote
Italian AI Evaluation & Annotation Specialist - Remote

AuraOne Human Data • USA

Remote
USD 28.000 - 48.000
Italian AI Model Review Analyst — Remote Contractor
Italian AI Model Review Analyst — Remote Contractor

AuraOne Human Data • USA

Remote
USD 34.000 - 55.000
Remote Italian Audiobook QA Specialist
Remote Italian Audiobook QA Specialist

Auraone • USA

Remote
USD 21.000 - 28.000
Bilingual Writer - Italian (Italy)
Bilingual Writer - Italian (Italy)

Auraone • USA

Remote
USD 18.000 - 26.000
Remote work
Italian-Language Business Analyst for AI Model Review
Italian-Language Business Analyst for AI Model Review

AuraOne Human Data • USA

Remote
USD 34.000 - 55.000
Remote Italian Language Expert for AI Training
Remote Italian Language Expert for AI Training

Remotebridge • USA

Remote
USD 34.000 - 76.000
Italian-Language Audio/Voice Annotation Trainer
Italian-Language Audio/Voice Annotation Trainer

AuraOne Human Data • USA

Remote
USD 28.000 - 48.000