Remote Frontier Model Evaluator: Wholesale & Tech Sales

AI Trainer Jobs; United States; Remote; USD 69,000 - 76,000; Full time

Benefits offered by this job

Remote work

US-eligible contractor

Competitive pay

Job summary

AuraOne is seeking a remote Wholesale and Technical Sales Reviewer to evaluate model outputs against a rubric, label issues, and provide structured feedback for retraining. This contractor role requires strong attention to detail and the ability to consistently apply rubrics across large batches.

Responsibilities include comparing responses, choosing the stronger answer with rationale, tagging content for safety and policy, and reporting rubric gaps to the program team.

Qualifications

• Experience evaluating wholesale and technical sales content.
• Ability applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Reliable async availability for at least 10 hours per week.
• Prior evaluation/annotation experience preferred.

Responsibilities

• Evaluate model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content with tags.
• Capture ambiguous prompts and route to program team for rubric updates.

Skills

Model evaluation

Rubric annotation

Severity tagging

Inter-rater calibration

Wholesale/Technical Sales

Job description

AuraOne is seeking a remote Wholesale and Technical Sales Reviewer to evaluate model outputs against a rubric, label issues, and provide structured feedback for retraining. This contractor role requires strong attention to detail and the ability to consistently apply rubrics across large batches.

Responsibilities include comparing responses, choosing the stronger answer with rationale, tagging content for safety and policy, and reporting rubric gaps to the program team.