Part-Time Medical AI Evaluation Specialist (MD/DO)

Mindrift; Clayton (MO); Remote; USD 83,000 - 179,000; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies. We are seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for an AI evaluation program.

This part-time, project-based work focuses on internal and general medicine, with tasks including scoring reasoning quality and writing concise rationales. You will perform multi-system vignette evaluations, apply evidence-based guidelines, and participate in blind reads with physician

Qualifications

• US-based physicians with MD or DO and active license.
• Board certification or residency completed with recent direct patient care.
• Multisystem diagnostic and treatment experience.
• Strong clinical reasoning and evidence-based guidelines application.
• Experience in medical AI evaluation or annotation is a plus.
• Excellent written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes with questions, proposed answers, and distractors across multiple medical domains.
• Score the clinical reasoning quality of benchmark items: vignette accuracy, completeness, and trap quality.
• Check whether the reasoning chain reaches the answer from vignette facts alone; write short rationales.
• Participate in three independent blind reads, followed by physician adjudication to ensure quality.

Skills

MD/DO

US medical license

Board certification

Clinical reasoning

Medical AI experience

English proficiency

Education

MD/DO degree

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies. We are seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for an AI evaluation program.

This part-time, project-based work focuses on internal and general medicine, with tasks including scoring reasoning quality and writing concise rationales. You will perform multi-system vignette evaluations, apply evidence-based guidelines, and participate in blind reads with physician