Physician AI Evaluation Reviewer (MD/DO) – Part-Time

Mindrift; Kenosha (WI); On-site; USD 58,000 - 96,000; Part time

Benefits offered by this job

Mindrift project-based work

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. We are seeking US-based, actively practicing physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program.

Contributors will score reasoning quality, assess vignette accuracy, determine trap quality, and write concise rationales.

Qualifications

• MD or DO with active, unrestricted US medical license.
• Board certification or residency with recent direct patient care.
• Broad, multi-system diagnostic and treatment experience.
• Strong clinical reasoning and evidence-based guidelines application.
• Prior experience in medical AI evaluation or medical annotation is a plus.
• Strong written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes paired with a question, a proposed answer, and a distractor.
• Score the clinical reasoning quality of benchmark items (accuracy, completeness, trap quality).
• Check whether the reasoning chain reaches the answer from vignette facts, with correct traces.
• Write short rationales to support scoring and be part of physician adjudication.
• Work within three independent blind reads, followed by physician adjudication.

Education

MD or DO degree

Active US medical license

Board certification or residency with direct patient care

Strong clinical reasoning

Excellent written English (C1+)

Medical AI evaluation experience

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. We are seeking US-based, actively practicing physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program.

Contributors will score reasoning quality, assess vignette accuracy, determine trap quality, and write concise rationales.