AI Video Quality & Craft Evaluator

Mercor; New York (NY); Remote; USD 34,000 - 83,000; Part time

Job summary

Mercor is seeking experienced film/theater and VFX professionals to join a cutting-edge AI evaluation team in New York. You will compare AI-generated video pairs, judge craft like lighting, framing, texture, and performance, and articulate concise reasons in a supervisor-friendly language.

This W-2, part-time role (about 20 hours/week) offers collaboration with cross-disciplinary experts, calibration rounds to ensure consistency, and ongoing rounds as models release.

Qualifications

• Experience in film/theater or related fields requiring meticulous visual judgment.
• Ability to commit at least 20 hours per week on weekdays.
• Fluency in English; proficiency in a second dubbing language is a strong plus.
• Attention to detail and the ability to describe judgments clearly.

Responsibilities

• Evaluate pairs of AI-generated videos made from the same prompt and choose the stronger result on craft axes.
• Write short, precise reasons for each judgment in supervisor language.
• Check each video against its prompt, including camera moves, framing or performance.
• Calibrate rounds with other raters to ensure consistency across the pool.
• Collaborate with other SMEs to ensure consistency and accuracy in evaluation data.

Education

Film/theater program + 2+ years experience

4+ years practical experience incl. two projects with theatrical release

Fluency in English

Job description

Mercor is seeking experienced film/theater and VFX professionals to join a cutting-edge AI evaluation team in New York. You will compare AI-generated video pairs, judge craft like lighting, framing, texture, and performance, and articulate concise reasons in a supervisor-friendly language.

This W-2, part-time role (about 20 hours/week) offers collaboration with cross-disciplinary experts, calibration rounds to ensure consistency, and ongoing rounds as models release.