PhD CS Expert — AI Model Evaluation (Remote, Up to $150/hr)

prolificacademicltd; United States; Remote; USD 34,000 - 207,000; Full time

Job summary

Prolific Academic Ltd is seeking highly credentialed individuals to help train and assess AI models by solving CS problems, narrating your reasoning aloud, and judging the quality of model answers.

You will record your screen, review AI-generated explanations, compare candidates, and provide improved feedback. Remote work is flexible; compensation is task-based and can reach up to $150 per hour.

Qualifications

• Hold, or be pursuing, a PhD in computer science or a related discipline.
• Demonstrate strong subject expertise and the ability to solve complex technical problems.
• Be comfortable thinking aloud and recording your task approach.
• Maintain close attention to detail during focused work, including sessions of up to an hour.
• Have reliable, fast internet access, a computer, and a PayPal account for payment.
• Pass a skills assessment for the AI tasks.

Responsibilities

• Work through challenging computer science problems while narrating your reasoning.
• Record your screen as you complete assigned tasks.
• Review AI-generated technical answers for correctness, depth, and quality of reasoning.
• Compare candidate answers, select the strongest, and explain the choice.
• Write improved examples, explanations, or structured feedback when responses need correction.

Skills

Think aloud

Problem solving

Attention to detail

Remote work discipline

Education

PhD in Computer Science

Job description

Prolific Academic Ltd is seeking highly credentialed individuals to help train and assess AI models by solving CS problems, narrating your reasoning aloud, and judging the quality of model answers.

You will record your screen, review AI-generated explanations, compare candidates, and provide improved feedback. Remote work is flexible; compensation is task-based and can reach up to $150 per hour.