Remote AI Model Evaluator: Engineering & Ops Quality

AI Trainer Jobs; United States; Remote; USD 110,000 - 165,000; Part time

Benefits offered by this job

Remote work

Flexible hours

Job summary

AI Trainer Jobs is seeking an Engineering / manufacturing / technical operations Evaluator to remotely review prompts and responses against AuraOne's quality rubric. You will compare paired outputs, label edge cases, and provide structured feedback the modeling team can use to retrain.

As a contractor, you will help Smith calibrate evaluations, identify failure modes, and write rationale across multiple batches with high attention to detail and clear communication.

Qualifications

• Must demonstrate careful judgment when evaluating model outputs against a rubric.
• Clear, concise written feedback and rationale are required.
• Experience with remote annotation or data labeling is advantageous.

Responsibilities

• Evaluate engineering / manufacturing / technical operations outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them for rubric updates.

Skills

Attention to detail

Written reasoning

Remote work experience

Inter-rater calibration

Job description

AI Trainer Jobs is seeking an Engineering / manufacturing / technical operations Evaluator to remotely review prompts and responses against AuraOne's quality rubric. You will compare paired outputs, label edge cases, and provide structured feedback the modeling team can use to retrain.

As a contractor, you will help Smith calibrate evaluations, identify failure modes, and write rationale across multiple batches with high attention to detail and clear communication.