Remote Javanese Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 34,000 - 55,000; Full time

Benefits offered by this job

Remote work

Contractor

Job summary

AuraOne is seeking a Javanese Language Expert for a remote evaluation track. Review prompts and responses, compare outputs, label edge cases, and generate structured feedback for the modeling team.

As a contractor, you will help turn evaluation outputs into auditable labels, rationales, and regression cases. Expect the role to involve evaluating frontier model outputs and calibrating evaluators.

Qualifications

• Experience evaluating language prompts using rubrics.
• Ability to annotate, label edge cases, and provide structured feedback.
• Proven written reasoning that clearly names issues and rubric clauses.

Responsibilities

• Evaluate model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and justify the chosen answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content with tags.
• Capture ambiguous prompts and route for rubric updates.
• Maintain reviewer-quality scores by calibration against gold-standard examples weekly.

Skills

Javanese language evaluation

Annotation/labeling

Inter-rater calibration

Written reasoning

Job description

AuraOne is seeking a Javanese Language Expert for a remote evaluation track. Review prompts and responses, compare outputs, label edge cases, and generate structured feedback for the modeling team.

As a contractor, you will help turn evaluation outputs into auditable labels, rationales, and regression cases. Expect the role to involve evaluating frontier model outputs and calibrating evaluators.