AI Training Evaluator for CS Grads — Remote & Flexible Pay

prolificacademicltd; United States; Remote; USD 50,000 - 83,000; Full time

Benefits offered by this job

Flexible, remote assignments

Skills assessment during screening

Expert participant network

Pay up to $60 per hour

Job summary

Prolific Academic Ltd seeks task-based evaluators to apply computer science knowledge to assess AI system outputs, review research material, and verify code segments. The role offers flexible, remote, hourly assignments with occasional longer blocks.

Candidates will rate accuracy, reasoning, and safety, review papers, check formulas, and provide expert feedback aligned with software engineering standards. A bachelor’s in CS and a PayPal account are required.

Qualifications

• Bachelor’s degree in computer science or a closely related technical field.
• Ability to interpret research papers, understand complex algorithms, and assess code logic.
• Strong analytical judgment and attention to technical errors or unsupported claims.
• A professional profile that can be used to verify education and background during screening.
• A PayPal account to receive payments.
• Ability to complete occasional tasks requiring one uninterrupted hour.

Responsibilities

• Compare AI responses to technical questions and rate their accuracy, reasoning, and safety.
• Review computer science research alongside AI-generated summaries and visual abstracts.
• Check technical claims, formulas, and findings against source material.
• Assess generated code and architectural diagrams for logical and structural correctness.
• Provide expert feedback that helps align AI outputs with professional software engineering and data science standards.

Skills

Analytical skills

Code review

Research interpretation

Quality assessment

Education

Bachelor's in Computer Science

Job description

Prolific Academic Ltd seeks task-based evaluators to apply computer science knowledge to assess AI system outputs, review research material, and verify code segments. The role offers flexible, remote, hourly assignments with occasional longer blocks.

Candidates will rate accuracy, reasoning, and safety, review papers, check formulas, and provide expert feedback aligned with software engineering standards. A bachelor’s in CS and a PayPal account are required.