Physician AI Evaluation Specialist (MD/DO) — Part-Time

Mindrift; Orangeburg (SC); Remote; USD 41,000 - 96,000; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. This opportunity involves evaluating clinical EHR vignettes for medical AI evaluation programs, with emphasis on internal and family medicine and related systems.

Contributors will perform tasks such as scoring reasoning quality, verifying chains of reasoning, and writing brief rationales.

Qualifications

• US-based physicians with an MD or DO and a current US license.
• Board certification or residency with recent direct patient care.
• Broad, multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine).
• Strong clinical reasoning and application of evidence-based guidelines.
• Prior experience in medical AI evaluation or medical annotation/QA is a plus.
• Strong written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes across diagnosis and treatment tasks.
• Score the clinical reasoning quality and trap quality.
• Check whether the reasoning chain reaches the answer from vignette facts and write short rationales.
• Work within three independent blind reads followed by physician adjudication.

Skills

Clinical reasoning

Strong written English

Medical AI evaluation (plus)

Education

MD or DO

Active US medical license

Board certification or residency

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. This opportunity involves evaluating clinical EHR vignettes for medical AI evaluation programs, with emphasis on internal and family medicine and related systems.

Contributors will perform tasks such as scoring reasoning quality, verifying chains of reasoning, and writing brief rationales.