AI Systems Evaluator - Remote, Pay-Per-Task

prolificacademicltd; United States; Remote; USD 55,000 - 83,000; Part time

Job summary

Prolific Academic Ltd is seeking individuals to evaluate AI systems through paid, project-based tasks. After a short skills assessment, applicants may review technical outputs and scientific material, with tasks varying in length and some requiring an uninterrupted hour.

Responsibilities include rating AI responses, verifying research summaries, and checking code and diagrams for accuracy. Remote participation, flexible workload, and PayPal payments are offered.

Qualifications

• Ability to interpret research papers and understand complex algorithms.
• Assess code logic and determine inaccuracies or flawed reasoning.
• Professional profile used for screening.
• PayPal account required to receive payments.

Responsibilities

• Compare AI responses to technical prompts and rate accuracy, reasoning, and safety.
• Check summaries of computer science research against original papers and related visuals.
• Identify errors in model interpretation of formulas, algorithms, or findings.
• Review generated code and system diagrams for logical and structural correctness.
• Provide expert feedback to align models with software engineering and data science standards.

Skills

Analytical thinking

Research interpretation

Code understanding

Education

Bachelor's degree in computer science or closely related technical field

Job description

Prolific Academic Ltd is seeking individuals to evaluate AI systems through paid, project-based tasks. After a short skills assessment, applicants may review technical outputs and scientific material, with tasks varying in length and some requiring an uninterrupted hour.

Responsibilities include rating AI responses, verifying research summaries, and checking code and diagrams for accuracy. Remote participation, flexible workload, and PayPal payments are offered.