Part-Time Physician AI Evaluator (MD/DO)

Mindrift; Madison (WI); Remote; USD 83,000 - 138,000; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on evaluating and improving AI systems. We are seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes, score clinical reasoning quality, and write succinct rationales.

This is a part-time, non-permanent project with pay per accepted task up to $100/hr. Strong English proficiency required; prior experience in medical AI evaluation preferred.

Qualifications

• Medical degree (MD/DO) required or currently practicing.
• Active, unrestricted US medical license required (verified with medical board).
• Broad, multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine).
• Strong clinical reasoning: differential diagnosis, next-step management, and evidence-based guidelines.
• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA is a plus.
• Strong written English (C1+) required.

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractors across multiple medical domains.
• Score the clinical reasoning quality of vignette items: accuracy, completeness, whether reasoning leads to the answer, and trap quality.
• Check whether reasoning traces from vignette facts reach the answer; write short rationales.
• Work within three independent blind reads, followed by physician adjudication.

Skills

Strong written English (C1+)

Strong clinical reasoning

Education

MD or DO

Board certification or residency completed

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on evaluating and improving AI systems. We are seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes, score clinical reasoning quality, and write succinct rationales.

This is a part-time, non-permanent project with pay per accepted task up to $100/hr. Strong English proficiency required; prior experience in medical AI evaluation preferred.