Remote QA Evaluator: Windows Premiere Apps

AI Trainer Jobs; United States; Remote; USD 41,000 - 55,000; Part time

Benefits offered by this job

Remote work

Contractor role

Competitive pay

Flexible tasks

Job summary

AI Trainer Jobs is seeking a remote contractor to evaluate application users premiere on Windows generalist evaluation outputs. You will apply a versioned rubric, label issues, and provide structured feedback for retraining efforts.

Responsibilities include comparing responses, selecting the stronger answer with justification, tagging policy violations, and documenting ambiguous prompts for rubric updates. This is a remote, part-time contractor position in the United States with hourly

Qualifications

• Prior evaluation, annotation, or human-rater experience in AI content review.
• Comfort applying rubrics across long batches.
• Clear written reasoning naming the issue and rubric clause.
• Strong attention to detail and ability to flag prompt problems.

Responsibilities

• Evaluate model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger one with a rationale.
• Label hallucinations, instruction-following failures, and unsafe content with tags.
• Capture ambiguous prompts and report feedback for rubric updates.

Skills

Evaluation experience

Written reasoning

Attention to detail

Flag prompts issues

Async availability

Education

Linguistics background

Job description

AI Trainer Jobs is seeking a remote contractor to evaluate application users premiere on Windows generalist evaluation outputs. You will apply a versioned rubric, label issues, and provide structured feedback for retraining efforts.

Responsibilities include comparing responses, selecting the stronger answer with justification, tagging policy violations, and documenting ambiguous prompts for rubric updates. This is a remote, part-time contractor position in the United States with hourly