Remote Evaluation Specialist: DaVinci Resolve (macOS)

AI Trainer Jobs; United States; Remote; USD 62,000 - 83,000; Full time

Job summary

AuraOne is seeking a remote evaluation specialist to review application-user outputs for the DaVinci Resolve on macOS Generalist track. You will compare paired responses, label edge cases, and provide structured feedback the modeling team can use to retrain.

As a contractor, you will help turn evaluation outputs into auditable labels, rationales, and regression cases. Expect to review frontier model outputs, judge benchmarks, and calibrate other evaluators.

Qualifications

• Prior evaluation, annotation, or human-rater experience in evaluation or related content.
• Ability to apply rubrics consistently across long batches.
• Clear written reasoning naming the issue and rubric clause.
• Strong attention to detail and flag prompts as problems.

Responsibilities

• Evaluate evaluation model outputs against rubric and assign severity tags.
• Compare paired responses and select stronger answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route to program team for rubric updates.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

DaVinci Resolve GA eval

Job description

AuraOne is seeking a remote evaluation specialist to review application-user outputs for the DaVinci Resolve on macOS Generalist track. You will compare paired responses, label edge cases, and provide structured feedback the modeling team can use to retrain.

As a contractor, you will help turn evaluation outputs into auditable labels, rationales, and regression cases. Expect to review frontier model outputs, judge benchmarks, and calibrate other evaluators.