AI Benchmark QC Auditor — Remote (North America)

Verita AI; United States; Remote; USD 9,100 - 23,000; Full time

Job summary

Verita AI is seeking a Coding & Rubric QC Expert (Pilot) to audit AI benchmark tasks. You will judge rubric clarity, verifiability, and alignment with the proposed gold standard, marking pass/fail with concise reasoning.

This independent contractor role is remote (North America-based), paid per completed task during a ~1-week pilot with ~10+ hours, and offers potential for continued or expanded work for strong performers. Tasks are paid weekly.

Qualifications

• Based in North America (remote work)
• Bachelor's degree in Computer Science or equivalent
• At least 3 months of human-data/annotation/AI-training work experience
• At least 1 year of software engineering experience

Responsibilities

• Review each rubric claim for clarity, specificity and verifiability.
• Mark each claim as pass, fail, or flagged for follow-up with a short reason.
• Spot missing connections between requirements and questions/answers.
• Double-check that the 'gold' answer is correct before treating it as ground truth.
• Provide an overall accept or reject decision with evidence-based feedback.

Skills

Human-data experience

Software engineering

Education

Bachelor's degree in Computer Science

Job description

Verita AI is seeking a Coding & Rubric QC Expert (Pilot) to audit AI benchmark tasks. You will judge rubric clarity, verifiability, and alignment with the proposed gold standard, marking pass/fail with concise reasoning.

This independent contractor role is remote (North America-based), paid per completed task during a ~1-week pilot with ~10+ hours, and offers potential for continued or expanded work for strong performers. Tasks are paid weekly.