German Audiobook QA Specialist - Remote

AuraOne; United States; On-site; USD 21,000 - 28,000; Part time

Job summary

AuraOne is seeking a Remote Audiobook QA Expert for German evaluation. You will review German generalist outputs, apply a versioned rubric, and label issues with structured feedback to aid retraining.

The contractor role emphasizes careful, documented reasoning and cross-checking with gold-standard samples. You will compare responses, choose the better answer with justification, and help identify recurring failure modes to improve the rubric and training iterations.

Qualifications

• Experience evaluating model outputs in German or adjacent content.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning citing rubric clauses.

Responsibilities

• Evaluate German generalist model outputs against a rubric and assign severity tags.
• Compare paired responses and select the stronger answer with a rationale.
• Label hallucinations, instruction-following failures, and unsafe content using structured tags.
• Capture ambiguous prompts and escalate to the program team for rubric updates.
• Calibrate reviewer quality against gold-standard examples weekly.
• Document recurring failure modes for modeling team feedback.

Skills

Model evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

German evaluation

Job description

AuraOne is seeking a Remote Audiobook QA Expert for German evaluation. You will review German generalist outputs, apply a versioned rubric, and label issues with structured feedback to aid retraining.

The contractor role emphasizes careful, documented reasoning and cross-checking with gold-standard samples. You will compare responses, choose the better answer with justification, and help identify recurring failure modes to improve the rubric and training iterations.