Part-Time Medical AI Evaluator (MD/DO) — EHR Vignettes

Mindrift; Alabama; On-site; USD 148,000 - 210,000; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This opportunity involves US-based physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program.

Participation is project-based and part-time. Physicians will evaluate vignettes, score reasoning quality, and write short rationales.

Qualifications

• MD or DO with active US license and board certification or recent residency
• Experience across internal, family, emergency, or hospital medicine
• Strong written English (C1+) and ability to review clinical vignettes with rationale
• Experience in medical AI evaluation or QA is a plus

Responsibilities

• Evaluate clinical EHR vignettes for AI evaluation tasks across multiple medical domains
• Score the clinical reasoning quality of benchmark items (diagnosis, treatment, and rationale)
• Check whether reasoning chains reach the answer from vignette facts and write short rationales
• Participate in three independent blind reads followed by physician adjudication

Skills

Clinical reasoning

US-based

English proficiency

Education

MD/DO degree

US medical license (active)

Board certification / residency completed

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This opportunity involves US-based physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program.

Participation is project-based and part-time. Physicians will evaluate vignettes, score reasoning quality, and write short rationales.