Physician AI Evaluator (MD/DO) - Part-Time Projects

Mindrift; Kentucky; Remote; USD 62,000 - 96,000; Full time

Job summary

Mindrift connects physicians with project-based AI evaluation work, offering flexible, US-based opportunities to test and improve clinical AI systems. This role focuses on evaluating EHR vignettes for internal medicine and related specialties, with tasks requiring careful clinical reasoning and documentation.

Expect 10–20 hours per week during active phases, paid per accepted task, with rates up to $70 per hour. This is a non-permanent, part-time engagement designed for practicing physicians.

Qualifications

• Medical degree (MD or DO) and an active, unrestricted US medical license (verified with the issuing medical board);
• Board certification or completed residency training, with recent direct patient care (attending-level preferred);
• Broad, multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine);
• Strong clinical reasoning: differential diagnosis, next-step management, and application of evidence-based guidelines;
• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA (a plus);
• Strong written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes with questions, proposed answers, and distractors across multiple medical systems.
• Score reasoning quality including vignette accuracy, completeness, and trap quality.
• Check that reasoning traces support answers and write short rationales.
• Work within three independent blind reads followed by physician adjudication.
• Contribute to improving AI tools through real clinical complexity.

Skills

Clinical reasoning

Medical AI evaluation

Strong written English

Education

MD/DO (Medical degree)

Job description

Mindrift connects physicians with project-based AI evaluation work, offering flexible, US-based opportunities to test and improve clinical AI systems. This role focuses on evaluating EHR vignettes for internal medicine and related specialties, with tasks requiring careful clinical reasoning and documentation.

Expect 10–20 hours per week during active phases, paid per accepted task, with rates up to $70 per hour. This is a non-permanent, part-time engagement designed for practicing physicians.