Remote AI Evaluator (CS Grad) - Model Reasoning, Up to $60/h

prolificacademicltd; United States; Remote; USD 68,000 - 97,000; Full time

Benefits offered by this job

Remote work

Flexible participation

Job summary

prolificacademic ltd is seeking candidates to evaluate AI systems through paid, project-based tasks. You will scrutinize model answers, research summaries, and generated technical artifacts for accuracy and sound reasoning.

A short skills test is part of screening; successful candidates may be invited to join available assignments. Responsibilities include ranking model responses, reviewing summaries, and validating code and diagrams for correctness.

Qualifications

• Bachelor's degree in computer science or related field.
• Ability to read research literature and understand complex technical concepts.
• Careful analytical judgment when assessing code, model reasoning, and technical claims.
• A professional profile for verification of educational and professional background during screening.
• A PayPal account for receiving payments.

Responsibilities

• Compare and rank model responses to technical prompts based on accuracy, logic, and safety.
• Review AI summaries and visuals alongside underlying computer science research.
• Identify incorrect readings of technical data, formulas, algorithms, or study findings.
• Check generated code examples and architecture diagrams for correctness.
• Provide specialist feedback aligning with software engineering and data science standards.

Skills

Analytical reasoning

Research literacy

Code and data review

Education

Bachelor's degree in computer science

Job description

prolificacademic ltd is seeking candidates to evaluate AI systems through paid, project-based tasks. You will scrutinize model answers, research summaries, and generated technical artifacts for accuracy and sound reasoning.

A short skills test is part of screening; successful candidates may be invited to join available assignments. Responsibilities include ranking model responses, reviewing summaries, and validating code and diagrams for correctness.