Thai Bilingual Evaluation Specialist (Remote)

AI Trainer Jobs; United States; Remote; USD 34,000 - 55,000; Full time

Job summary

AuraOne is seeking a Thai bilingual expert to evaluate generalist evaluation prompts and responses remotely. You will compare outputs, label edge cases, and produce structured feedback to retrain models. This contractor role supports auditability of labels and rationale for QA.

Applicants should have linguistic or trust & safety review background and be able to work asynchronously. Responsibilities include rating model outputs, tagging content with policy categories, and calibrating against gold

Qualifications

• Prior evaluation, annotation, or human-rater experience on thai generalist evaluation or adjacent content for Thai Bilingual Expert work.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Experience with inter-rater agreement metrics and calibration cycles.
• Background in linguistics, content moderation, or trust & safety review.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate thai generalist evaluation model outputs against a versioned rubric and assign severity tags for Thai Bilingual Expert assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.

Skills

Thai generalist evaluation

Inter-rater calibration

Rubric-based annotation

Severity tagging

Structured feedback writing

Job description

AuraOne is seeking a Thai bilingual expert to evaluate generalist evaluation prompts and responses remotely. You will compare outputs, label edge cases, and produce structured feedback to retrain models. This contractor role supports auditability of labels and rationale for QA.

Applicants should have linguistic or trust & safety review background and be able to work asynchronously. Responsibilities include rating model outputs, tagging content with policy categories, and calibrating against gold