Physician AI Reviewer (MD/DO) — Part-Time, Project-Based

Mindrift; Boulder (CO); On-site; USD 55,000 - 138,000; Part time

Benefits offered by this job

Up to $100/hr

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. We are seeking US-based, practicing physicians to evaluate clinical EHR vignettes for a medical AI evaluation program. This is part-time and project-based.

Contributors will assess vignette realism, score clinical reasoning quality, and provide concise rationales. Tasks are paid per accepted task, up to $100/hr.

Qualifications

• Medical degree (MD or DO) and an active, unrestricted US medical license (verified with the issuing medical board);
• Board certification or completed residency training, with recent direct patient care (attending-level preferred);
• Broad, multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine);
• Strong clinical reasoning: differential diagnosis, next-step management, and application of evidence-based guidelines;
• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA (a plus);
• Strong written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes paired with a question, a proposed answer, and a distractor "trap" answer across diagnosis and treatment tasks, spanning cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems;
• Score the clinical reasoning quality of benchmark items: vignette accuracy and completeness, whether the vignette gives the answer away, answer correctness and gradeability, and trap quality;
• Check whether the reasoning chain reaches the answer from vignette facts alone, correct reasoning traces, and write short rationales;
• Work within three independent blind reads, followed by physician adjudication. Real clinical complexity only. You're improving the AI tools you'll eventually use yourself

Skills

MD/DO

Active US license

Board certification

Clinical reasoning

Medical annotation experience

Strong written English

Education

Medical degree (MD or DO)

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. We are seeking US-based, practicing physicians to evaluate clinical EHR vignettes for a medical AI evaluation program. This is part-time and project-based.

Contributors will assess vignette realism, score clinical reasoning quality, and provide concise rationales. Tasks are paid per accepted task, up to $100/hr.