Part-Time AI Medical Evaluator (MD/DO)

Mindrift; Oklahoma; Remote; USD 41,000 - 96,000; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This US-based, part-time role invites physicians to evaluate clinical EHR vignettes and score reasoning quality across multiple medical domains.

You’ll work in a rigorous, blinded process with physician adjudication to help advance real-world AI tools. Qualifications include MD/DO with an active US license, board certification or residency,

Qualifications

• MD or DO with active, unrestricted US medical license (verified with medical board).
• Board certification or completed residency with recent direct patient care.
• Broad, multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine).
• Strong clinical reasoning and application of evidence-based guidelines.
• Experience in medical AI evaluation or related QA is a plus.
• Strong written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes paired with a question, a proposed answer, and a distractor “trap” answer across diagnosis and treatment tasks across multiple systems.
• Score the clinical reasoning quality of benchmark items: vignette accuracy, completeness, trap quality, and adducive reasoning.
• Check whether the reasoning chain reaches the answer from vignette facts alone and write short rationales.
• Work within three independent blind reads, followed by physician adjudication.

Skills

Clinical reasoning

Written English

Medical license

Education

MD or DO

Active US medical license

Board certification or residency

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This US-based, part-time role invites physicians to evaluate clinical EHR vignettes and score reasoning quality across multiple medical domains.

You’ll work in a rigorous, blinded process with physician adjudication to help advance real-world AI tools. Qualifications include MD/DO with an active US license, board certification or residency,