AI Medical Evaluator (MD/DO) - Part-Time Project

Mindrift; Guin (AL); Remote; USD 83,000 - 179,000; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. We’re looking for US-based, actively practicing physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program.

Tasks involve evaluating vignettes, scoring reasoning quality, and writing short rationales across multiple systems. Work is part-time, non-permanent, with compensation up to $130/hr per accepted task.

Qualifications

• Physician with MD or DO and active US medical license.
• Board certification or completed residency training with recent direct patient care.
• Broad, multi-system diagnostic and treatment experience.
• Strong clinical reasoning and knowledge of guidelines.
• Prior experience in medical AI evaluation or annotation is a plus.
• Excellent written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes with questions and distractors across domains.
• Score clinical reasoning quality of benchmark items (accuracy, completeness, trap quality).
• Check whether reasoning chains reach the answer from vignette facts and write rationales.
• Participate in three blind reads followed by physician adjudication.

Skills

Clinical reasoning

Written English

US medical license

Education

MD or DO

Active US medical license

Board certification or residency training

Tools

Medical AI evaluation

Clinical content review

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. We’re looking for US-based, actively practicing physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program.

Tasks involve evaluating vignettes, scoring reasoning quality, and writing short rationales across multiple systems. Work is part-time, non-permanent, with compensation up to $130/hr per accepted task.