Remote Bilingual Spanish AI Quality Evaluator

AuraOne; United States; On-site; USD 27,552 - 55,104; Full time

Job summary

AuraOne is seeking a Bilingual Spanish AI Evaluation Specialist to work remotely from the United States as a contractor. You will evaluate Spanish generalist outputs, apply a versioned rubric, and tag edge cases with structured labels.

You will compare paired responses and provide concise rationales to guide model retraining. Responsibilities include maintaining calibration against gold-standard examples, identifying recurring failures, and routing ambiguous prompts for rubric updates.

Qualifications

• Prior evaluation, annotation, or human-rater experience on spanish generalist evaluation or adjacent content.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Ability to apply multi-page rubrics consistently across long batches.

Responsibilities

• Evaluate spanish generalist evaluation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and provide a written rationale for the stronger answer.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back for rubric updates.
• Maintain reviewer quality scores and calibrate against gold-standard examples weekly.
• Document recurring failure modes to guide training.

Skills

Model evaluation

Rubric annotation

Severity tagging

Inter-rater calibration

Spanish evaluation

Job description

AuraOne is seeking a Bilingual Spanish AI Evaluation Specialist to work remotely from the United States as a contractor. You will evaluate Spanish generalist outputs, apply a versioned rubric, and tag edge cases with structured labels.

You will compare paired responses and provide concise rationales to guide model retraining. Responsibilities include maintaining calibration against gold-standard examples, identifying recurring failures, and routing ambiguous prompts for rubric updates.