Remote Korean Language Evaluation Specialist (AI Quality)

AuraOne; United States; Remote; USD 34,000 - 55,000; Full time

Job summary

AuraOne is seeking a Business Document Expert (Korean Speaker) to remotely evaluate prompts and responses against our quality rubric. You will compare outputs, label edge cases, and draft structured feedback to retrain the model.

You will review frontier model outputs, judge benchmark failures, and help calibrate other evaluators. This is a contractor role with flexible hours and US eligibility for remote work.

Qualifications

• Experience evaluating Korean generalist prompts and responses.
• Ability to write structured feedback for retraining.
• Familiarity with rubrics and edge-case labeling.

Responsibilities

• Evaluate Korean generalist evaluation model outputs against a rubric.
• Compare paired responses and select the stronger answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content.
• Capture ambiguous prompts and suggest rubric updates.
• Calibrate reviewer scores against gold-standard examples weekly.

Skills

Korean language

Evaluation/annotation experience

Structured feedback writing

Attention to detail

Async work availability

Job description

AuraOne is seeking a Business Document Expert (Korean Speaker) to remotely evaluate prompts and responses against our quality rubric. You will compare outputs, label edge cases, and draft structured feedback to retrain the model.

You will review frontier model outputs, judge benchmark failures, and help calibrate other evaluators. This is a contractor role with flexible hours and US eligibility for remote work.