Remote Indonesian Evaluation Psychologist (PhD) – QA & Feedback

AuraOne, Inc.; United States; À distance; USD 34 000 - 69 000; Temps partiel

Résumé du poste

This contractor role requires clear written reasoning, attention to detail, and reliable async availability for at least 10 hours per week. Applicants must be authorized to work from the US; compensation is hourly and confirmed after the interview.

Qualifications

• Experience in evaluating Indonesian generalist evaluation or related content.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear, written reasoning that names the issue and the rubric clause.
• Attention to detail and ability to flag when prompts are problematic.

Responsabilités

• Evaluate Indonesian generalist evaluation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and escalate to the program team for rubric updates.
• Maintain reviewer quality scores by calibrating against gold-standard examples weekly.
• Document recurring failure modes to inform future training.”

Connaissances

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Indonesian generalist evaluation

Description du poste

This contractor role requires clear written reasoning, attention to detail, and reliable async availability for at least 10 hours per week. Applicants must be authorized to work from the US; compensation is hourly and confirmed after the interview.