AI Model Evaluator — CS Graduate (Remote, Flexible)

prolificacademicltd; United States; Remote; USD 68,000 - 97,000; Full time

Benefits offered by this job

Remote work

Flexible hours

Job summary

prolificacademicltd offers paid, task-based roles evaluating AI outputs. You will rank responses to technical prompts, verify summaries against CS papers, and audit code or diagrams for soundness.

A brief assessment screens applicants, and selected participants join projects as available. Responsibilities include examining reasoning quality, ensuring accuracy of research interpretations, and providing expert feedback to align models with software engineering and data science standards.

Qualifications

• Bachelor’s degree in computer science or related technical field.
• Ability to read research papers and understand algorithms.
• Strong analysis skills including reviewing code logic.
• Professional profile verification during screening.
• PayPal account required for payments.

Responsibilities

• Rank alternative AI responses to technical prompts for correctness, reasoning quality, and safety.
• Check model-written summaries and graphical abstracts against computer science research papers.
• Find errors in interpretations of technical evidence, mathematical formulas, algorithms, or research findings.
• Audit generated code and architectural diagrams for sound structure and function.
• Provide expert feedback to guide models toward professional standards in software engineering and data science.

Skills

Strong analytical skills

Understand research papers

Code review

Education

Bachelor's degree in computer science or related field

Job description

prolificacademicltd offers paid, task-based roles evaluating AI outputs. You will rank responses to technical prompts, verify summaries against CS papers, and audit code or diagrams for soundness.

A brief assessment screens applicants, and selected participants join projects as available. Responsibilities include examining reasoning quality, ensuring accuracy of research interpretations, and providing expert feedback to align models with software engineering and data science standards.