Part-Time Medical AI Evaluator (MD/DO)

Mindrift; Kansas City (MO); On-site; USD 114,000 - 162,000; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. This opportunity seeks US-based physicians to evaluate clinical EHR vignettes for a medical AI evaluation program.

Contributors will score clinical reasoning, check whether the reasoning traces lead to the correct conclusions, and write brief rationales. This is a part-time, non-permanent project with payment up to $100/hr per accepted task.

Qualifications

• MD or DO with active, unrestricted US medical license
• Board certification or residency training with recent direct patient care
• Broad diagnostic and treatment experience across internal, family, emergency, or hospital medicine
• Strong clinical reasoning and application of evidence-based guidelines
• Experience in medical AI evaluation or exam-question review is a plus
• Strong written English (C1+)

Responsibilities

• Evaluate clinical EHR vignettes with questions, proposed answers, and distractors
• Score clinical reasoning quality and vignette accuracy
• Check reasoning chains and write short rationales
• Participate in independent blind reads with physician adjudication

Skills

Strong written English

Clinical reasoning

Medical AI evaluation experience

Education

MD or DO

Board certification or residency training

Tools

Annotation/QA experience

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. This opportunity seeks US-based physicians to evaluate clinical EHR vignettes for a medical AI evaluation program.

Contributors will score clinical reasoning, check whether the reasoning traces lead to the correct conclusions, and write brief rationales. This is a part-time, non-permanent project with payment up to $100/hr per accepted task.