Blender on Windows QA Evaluator - Remote (Contractor)

AuraOne; United States; On-site; USD 41,000 - 55,000; Part time

Job summary

AuraOne is seeking a remote evaluator to review application users blender on windows generalist evaluation outputs. You will apply a versioned rubric to label, justify, and calibrate responses for auditability and retraining purposes.

Responsibilities include comparing responses, tagging issues, and documenting recurring failure modes. This role requires strong attention to detail, clear written reasoning, and reliable weekly availability.

Qualifications

• Experience evaluating or annotating content.
• Ability to apply long rubrics consistently across batches.
• Clear written reasoning naming the issue and rubric clause.
• Attention to detail and ability to flag prompt problems.
• Reliable availability for at least 10 hours per week.

Responsibilities

• Evaluate model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and provide a rationale for the stronger answer.
• Label hallucinations, instruction-following failures, and unsafe content with tags.
• Capture ambiguous prompts and route to rubric updates.
• Calibrate reviewer quality against gold-standard examples weekly.
• Document recurring failure modes for future training.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Application Users Blender on Windows

Job description

AuraOne is seeking a remote evaluator to review application users blender on windows generalist evaluation outputs. You will apply a versioned rubric to label, justify, and calibrate responses for auditability and retraining purposes.

Responsibilities include comparing responses, tagging issues, and documenting recurring failure modes. This role requires strong attention to detail, clear written reasoning, and reliable weekly availability.