Medical AI Evaluation Specialist (MD/DO) - Part-Time

Mindrift; Minneapolis (MN); Remote; USD 83,000 - 138,000; Part time

Job summary

Mindrift is seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program. Work is project-based and involves scoring reasoning quality across multiple organ systems.

Contributors typically engage part-time with flexible hours, receiving compensation per accepted task up to $100/hr. Strong English writing is required and prior AI evaluation experience is a plus.

Qualifications

• Medical degree (MD or DO) and an active, unrestricted US medical license.
• Board certification or completed residency training, with recent direct patient care.
• Broad, multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine).
• Strong clinical reasoning: differential diagnosis, next-step management, and guidelines application.
• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA.
• Strong written English (C1+)

Responsibilities

• Evaluate clinical EHR vignettes with questions, proposed answers, and distractor options across multiple organ systems.
• Score clinical reasoning quality: vignette accuracy, completeness, and trap quality.
• Ensure reasoning derives answers from vignette facts and write concise rationales.
• Work within three blind reads followed by physician adjudication.
• Contribute to improving AI evaluation tools and processes.

Skills

MD/DO degree

US medical license

Board certification

Residency training

Clinical reasoning

Medical AI evaluation experience

Strong English writing

Education

Medical degree (MD/DO)

Job description

Mindrift is seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program. Work is project-based and involves scoring reasoning quality across multiple organ systems.

Contributors typically engage part-time with flexible hours, receiving compensation per accepted task up to $100/hr. Strong English writing is required and prior AI evaluation experience is a plus.