Remote AI Evaluation Scientist | Flexible Hours

prolificacademicltd; United States; Remote; USD 68,000 - 97,000; Full time

Job summary

Prolific Academic Ltd offers remote, project-based tasks to contribute computer science expertise for AI system training and evaluation. Tasks may include judging model answers, reviewing code and diagrams, and assessing technical summaries.

A short skills assessment is part of screening, with availability depending on researchers requesting relevant expertise. Pay is set by the requesting research project and can reach up to $60 per hour, with shorter tasks available.

Qualifications

• Bachelor's degree in computer science or a closely related technical field.
• Ability to read research papers, understand complex algorithms, and assess code logic.
• Strong analytical judgment and attention to technical inaccuracies or flawed reasoning.
• A professional profile that can be used to verify education and background during screening.
• A payment account for receiving task compensation.
• Availability for some assignments requiring one uninterrupted hour.

Responsibilities

• Compare AI responses to technical questions and rate their accuracy, reasoning, and safety.
• Review computer science papers and AI-generated summaries or graphical abstracts for fidelity to the research.
• Find errors in explanations of technical data, formulas, algorithms, and research results.
• Provide expert judgments that help improve model behavior in software engineering and data science.
• Check generated code and architecture diagrams for logical and structural correctness.

Skills

Reading research papers

Analytical judgment

Attention to technical inaccuracies

Education

Bachelor's degree in computer science

Job description

Prolific Academic Ltd offers remote, project-based tasks to contribute computer science expertise for AI system training and evaluation. Tasks may include judging model answers, reviewing code and diagrams, and assessing technical summaries.

A short skills assessment is part of screening, with availability depending on researchers requesting relevant expertise. Pay is set by the requesting research project and can reach up to $60 per hour, with shorter tasks available.