Medical AI Evaluation Physician (MD/DO) - Part-Time

Mindrift; Mississippi; Remote; USD 58,000 - 96,000; Part time

Job summary

Mindrift connects physicians with project-based AI evaluation opportunities for leading tech companies, focusing on testing and improving AI systems. We seek US-based MDs or DOs to evaluate clinical EHR vignettes across internal and family medicine.

Tasks include scoring reasoning quality, verifying vignette consistency, and drafting brief rationales for adjudication. Compensation is per accepted task, with rates up to $70 per hour depending on qualification tier and efficiency.

Qualifications

• MD or DO degree required with active US medical license
• Board certification or completed residency with recent direct patient care
• Broad multi-system diagnostic and treatment experience (internal, family, emergency, hospital medicine)
• Strong clinical reasoning and application of guidelines
• Experience in medical AI evaluation or medical annotation is a plus
• Strong written English (C1+)

Responsibilities

• Evaluate clinical EHR vignettes with questions, proposed answers, and distractors across multiple medical systems
• Score clinical reasoning quality: vignette accuracy, completeness, trap quality, and whether reasoning leads to the correct answer
• Check whether the reasoning chain can be traced from vignette facts and write brief rationales
• Participate in blind reads and physician adjudication to improve AI evaluation tools

Skills

Strong written English

Education

MD or DO degree

Active US medical license

Board certification or completed residency

Job description

Mindrift connects physicians with project-based AI evaluation opportunities for leading tech companies, focusing on testing and improving AI systems. We seek US-based MDs or DOs to evaluate clinical EHR vignettes across internal and family medicine.

Tasks include scoring reasoning quality, verifying vignette consistency, and drafting brief rationales for adjudication. Compensation is per accepted task, with rates up to $70 per hour depending on qualification tier and efficiency.