Remote Bilingual Korean AI Quality Evaluator

AI Trainer Jobs; United States; Remote; USD 34,000 - 55,000; Part time

Benefits offered by this job

Remote work

Job summary

AuraOne is seeking a Bilingual Korean AI Evaluation Specialist to remotely review Korean prompts and responses using AuraOne's rubric. You will compare paired outputs, label edge cases, and provide structured feedback for retraining the model.

The role involves turning evaluation outputs into auditable labels, rationale, and regression cases for Human Data workflows. Track evaluation, judge benchmark failures, and calibrate other evaluators as part of a remote contractor team.

Qualifications

• Experience evaluating AI model outputs against rubrics.
• Ability to label edge cases with structured tags.
• Strong written reasoning to name issues and rubric clauses.
• Familiarity with bilingual Korean content evaluation.

Responsibilities

• Evaluate Korean generalist outputs against a versioned rubric.
• Compare paired responses and justify the stronger one.
• Label hallucinations and unsafe content with tags.
• Capture ambiguous prompts and route to rubric updates.
• Calibrate reviewer quality against gold-standard examples weekly.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Korean generalist evaluation

Job description

AuraOne is seeking a Bilingual Korean AI Evaluation Specialist to remotely review Korean prompts and responses using AuraOne's rubric. You will compare paired outputs, label edge cases, and provide structured feedback for retraining the model.

The role involves turning evaluation outputs into auditable labels, rationale, and regression cases for Human Data workflows. Track evaluation, judge benchmark failures, and calibrate other evaluators as part of a remote contractor team.