Remote Cultural Context Reviewer for Low-Resource Languages

AuraOne, Inc.; United States; Remote; USD 34.000 - 83.000; Teilzeit

Benefits dieser Stelle

Contractor engagement

Zusammenfassung

AuraOne, Inc. is seeking a remote Low-Resource Language Cultural Context Reviewer to evaluate prompts and responses against its quality rubric. You will compare paired outputs, label edge cases, and write structured feedback for retraining.

As a contractor, you will work remotely with US eligibility, reviewing and tagging outputs, keeping calibration with gold standards, and reporting recurring failures to improve future training runs.

Qualifikationen

• Prior evaluation/annotation experience in low-resource language contexts.
• Ability to apply multi-page rubrics consistently.
• Clear written reasoning citing rubric clause being applied.
• Strong attention to detail and flagging when a prompt is the problem.
• Reliable async availability for at least 10 hours per week.

Aufgaben

• Evaluate model outputs against rubric and assign severity tags.
• Compare paired responses and select stronger answer with rationale.
• Label unsafe content with the correct policy category and severity.
• Audit batches and report rubric drift to the program lead.
• Propose rubric clarifications after spotting recurring failures.
• Calibrate reviewer quality against gold-standard examples weekly.

Kenntnisse

Evaluation experience

Rubric-based annotation

Inter-rater calibration

Attention to detail

Async availability

Ausbildung

Linguistics background

Jobbeschreibung

AuraOne, Inc. is seeking a remote Low-Resource Language Cultural Context Reviewer to evaluate prompts and responses against its quality rubric. You will compare paired outputs, label edge cases, and write structured feedback for retraining.

As a contractor, you will work remotely with US eligibility, reviewing and tagging outputs, keeping calibration with gold standards, and reporting recurring failures to improve future training runs.