Physician AI Evaluation Specialist (MD/DO) – Part-Time

Mindrift; Town of King (WI); Remote; USD 80,000 - 113,000; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing and evaluating AI systems. We are seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for an AI evaluation program, mainly in internal and general medicine.

You will score clinical reasoning quality, assess vignette accuracy, and write concise rationales. Tasks are paid per accepted item, with potential up to $70/hr, and require 10–20 hours per week during active

Qualifications

• MD or DO with active US medical license.
• Board certification or residency training.
• Broad multi-system diagnostic and treatment experience.
• Strong clinical reasoning and evidence-based guidelines.
• Prior experience in medical AI evaluation or medical annotation is a plus.
• Strong written English proficiency.

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractors.
• Score the clinical reasoning quality of benchmark items including accuracy and trap quality.
• Check whether the reasoning chain reaches the answer from vignette facts and write short rationales.
• Work within three blind reads followed by physician adjudication.

Skills

Clinical reasoning

Medical knowledge

English writing

Education

MD or DO

Active US medical license

Board certification or residency training

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing and evaluating AI systems. We are seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for an AI evaluation program, mainly in internal and general medicine.

You will score clinical reasoning quality, assess vignette accuracy, and write concise rationales. Tasks are paid per accepted item, with potential up to $70/hr, and require 10–20 hours per week during active