Clinical AI Evaluator (MD/DO) – Part-Time Projects

Mindrift; Central City (KY); Remote; USD 58,000 - 96,000; Part time

Job summary

Mindrift is seeking US-based physicians to evaluate clinical EHR vignettes for AI evaluation projects. Tasks are part-time and project-based, requiring meticulous review of vignette questions, answers, and rationales.

You will participate in blind reads and physician adjudication to improve AI tools used in healthcare. Ideal candidates hold an MD/DO with an active license, board certification or residency, broad multi-system experience, and strong English writing skills.

Qualifications

• MD or DO with active US medical license.
• Board certification or completed residency training, with recent direct patient care.
• Broad, multi-system diagnostic and treatment experience.
• Strong clinical reasoning and application of evidence-based guidelines.
• Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA (a plus).
• Strong written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes with questions, proposed answers, and distractors across various systems.
• Score the clinical reasoning quality: vignette accuracy, completeness, whether the vignette leads to the answer, and trap quality.
• Check whether reasoning traces reach the answer from vignette facts, and write short rationales.
• Participate in three independent blind reads, followed by physician adjudication.

Skills

Strong written English (C1+)

Clinical reasoning

Attention to detail

Education

MD or DO degree

Active US medical license

Board certification or residency training

Tools

Job description

Mindrift is seeking US-based physicians to evaluate clinical EHR vignettes for AI evaluation projects. Tasks are part-time and project-based, requiring meticulous review of vignette questions, answers, and rationales.

You will participate in blind reads and physician adjudication to improve AI tools used in healthcare. Ideal candidates hold an MD/DO with an active license, board certification or residency, broad multi-system experience, and strong English writing skills.