Remote AI Benchmark QC & Coding Auditor

Remolity, Inc.; United States; Remote; USD 21,000 - 34,000; Part time

Benefits offered by this job

Paid pilot

Flexible scheduling

Remote work within eligible countries

Job summary

Verita AI is seeking Coding & QC Experts for a paid pilot evaluating the quality of AI benchmark tasks. Each task includes a question, a proposed gold answer and a rubric of claims describing what a correct answer should contain.

Critically audit existing work, assess rubric claims, and verify the reference answer before treating it as ground truth. This evaluative work suits reviewers who enjoy code review, QA, and auditing others' work.

Qualifications

• Bachelor's degree in Computer Science or equivalent.
• At least 3 months of human-data, annotation or AI-training experience.
• At least 1 year of software engineering experience outside human-data work.
• Based in United States or Canada.

Responsibilities

• Review every rubric claim for clarity, specificity and verifiability.
• Mark claims as pass, fail or flagged for follow-up with a short written reason.
• Identify requirements that do not connect to the question or proposed answer.
• Check the correctness of the proposed gold answer before treating it as ground truth.
• Provide an overall accept or reject decision for each task, supported by findings.

Skills

Code review

Quality assurance

Auditing

Education

Bachelor's degree in Computer Science

Job description

Verita AI is seeking Coding & QC Experts for a paid pilot evaluating the quality of AI benchmark tasks. Each task includes a question, a proposed gold answer and a rubric of claims describing what a correct answer should contain.

Critically audit existing work, assess rubric claims, and verify the reference answer before treating it as ground truth. This evaluative work suits reviewers who enjoy code review, QA, and auditing others' work.