Part-Time Medical AI Evaluator (MD/DO)

Mindrift; Denver (CO); Remote; USD 83,000 - 138,000; Part time

Job summary

Mindrift is seeking US-based physicians to participate in a project-based AI evaluation program. You will evaluate clinical EHR vignettes across multiple medical systems, scoring reasoning quality and ensuring conclusions derive from vignette facts.

Short rationales accompany assessments, with adjudication after three blind reads. Applicants should hold an MD/DO with an active US license, board certification or residency, and demonstrate strong English writing.

Qualifications

• MD or DO degree and active US medical license required.
• Board certification or residency training with recent direct patient care preferred.
• Broad multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine).
• Strong clinical reasoning and application of evidence-based guidelines.
• Prior experience in medical AI evaluation or medical annotation/QA is a plus.
• Strong written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractors across multiple body systems.
• Score the clinical reasoning quality, vignette accuracy, completeness, and trap quality.
• Check whether reasoning reaches the answer from vignette facts and write short rationales.
• Work within three independent blind reads, followed by physician adjudication.

Skills

Clinical reasoning

Written English

Education

MD or DO

Active US license

Board certification or residency

Job description

Mindrift is seeking US-based physicians to participate in a project-based AI evaluation program. You will evaluate clinical EHR vignettes across multiple medical systems, scoring reasoning quality and ensuring conclusions derive from vignette facts.

Short rationales accompany assessments, with adjudication after three blind reads. Applicants should hold an MD/DO with an active US license, board certification or residency, and demonstrate strong English writing.