Remote Korean AI Evaluation & Feedback Specialist

AuraOne, Inc.; United States; Remote; USD 57.000 - 81.000; Vollzeit

Zusammenfassung

AuraOne, Inc. is seeking a contractor-level reviewer for a remote evaluation track focused on Korean generalist prompts and responses. You will assess model outputs against a quality rubric, annotate with severity, and provide rationale to guide improvements.

Ideal candidates have prior evaluation experience and can apply rubrics across long batches, with clear reasoning and reliable weekly availability. Remote US-eligible contractor role with $50 per hour compensation.

Qualifikationen

• Prior evaluation, annotation, or human-rater experience on korean generalist evaluation or adjacent content for Korean ChatGPT User work.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Strong attention to detail and the ability to flag when a prompt itself is the problem.
• Reliable async availability for at least 10 hours per week.

Aufgaben

• Evaluate korean generalist evaluation model outputs against a versioned rubric and assign severity tags for Korean ChatGPT User assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.
• Document recurring failure modes so the modeling team can target them in the next training run.

Kenntnisse

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Korean generalist evaluation

Eligibility

Jobbeschreibung

AuraOne, Inc. is seeking a contractor-level reviewer for a remote evaluation track focused on Korean generalist prompts and responses. You will assess model outputs against a quality rubric, annotate with severity, and provide rationale to guide improvements.

Ideal candidates have prior evaluation experience and can apply rubrics across long batches, with clear reasoning and reliable weekly availability. Remote US-eligible contractor role with $50 per hour compensation.