Medical AI Evaluator (MD/DO) – Part-time

Mindrift; Clayton (NC); Remote; USD 55,000 - 96,000; Part time

Job summary

Mindrift connects specialists with AI-driven evaluation projects for leading tech companies, focusing on testing and improving AI systems. We seek US-based physicians to evaluate clinical EHR vignettes and provide structured rationales, with adjudication across multiple rounds.

This part-time, project-based work offers flexible hours and compensation per task up to $70/hr. Ideal candidates will hold an MD/DO and an active US license, with recent direct patient care and strong written English.

Qualifications

• MD or DO with active US license
• Board certification or residency with recent direct patient care
• Broad multi-system diagnostic and treatment experience
• Strong clinical reasoning and evidence-based practice
• Experience in medical AI evaluation or QA is a plus
• Excellent written English (C1+)

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractors across multiple systems
• Score clinical reasoning quality, vignette accuracy, completeness, and trap quality
• Check whether reasoning traces from vignette facts lead to the answer and write short rationales
• Participate in three blind reads followed by physician adjudication

Skills

Clinical reasoning

Attention to detail

English proficiency (written)

Education

MD or DO

Active US medical license

Board certification or residency with recent patient care

Job description

Mindrift connects specialists with AI-driven evaluation projects for leading tech companies, focusing on testing and improving AI systems. We seek US-based physicians to evaluate clinical EHR vignettes and provide structured rationales, with adjudication across multiple rounds.

This part-time, project-based work offers flexible hours and compensation per task up to $70/hr. Ideal candidates will hold an MD/DO and an active US license, with recent direct patient care and strong written English.