Part-Time Medical AI Evaluator (MD/DO)

Mindrift; Minneota (MN); Remote; USD 69,000 - 96,000; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on evaluating clinical AI systems. We seek US-based physicians (MD/DO) to evaluate EHR vignettes, score reasoning quality, and provide concise rationales.

Projects are part-time and not permanent, with a bias toward practicing clinicians. Ideal candidates hold an active US medical license, board certification or recent residency, and strong English writing skills.

Qualifications

• Medical degree (MD or DO) and active US license.
• Board certification or completed residency training with recent direct patient care.
• Broad multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine).
• Strong clinical reasoning and application of evidence-based guidelines.
• Experience in medical AI evaluation, clinical content, or exam-question review is a plus.
• Strong written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractors across multiple organ systems.
• Score clinical reasoning quality, vignette accuracy, completeness, and trap quality.
• Check whether reasoning traces correctly derive answers from vignette facts and write short rationales.
• Collaborate in three blind reads and physician adjudication to improve AI evaluation quality.

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on evaluating clinical AI systems. We seek US-based physicians (MD/DO) to evaluate EHR vignettes, score reasoning quality, and provide concise rationales.

Projects are part-time and not permanent, with a bias toward practicing clinicians. Ideal candidates hold an active US medical license, board certification or recent residency, and strong English writing skills.