Physician AI Evaluation Specialist (MD/DO)

Mindrift; Central City (CO); On-site; USD 69,000 - 96,000; Part time

Benefits offered by this job

Per-task payment up to $70/hr

Flexible project hours (10–20 hrs/week

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. This opportunity involves evaluating medical AI benchmarks and providing structured feedback on clinical vignettes and reasoning.

We are seeking US-based physicians (MD/DO) with active licenses to participate part-time. Tasks are paid per accepted item, with potential rates up to $70 per hour depending on qualification tier and efficiency.

Qualifications

• Medical degree (MD or DO) and an active US medical license.
• Board certification or completed residency with recent direct patient care.
• Broad, multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine).
• Strong clinical reasoning: differential diagnosis, next-step management, and evidence-based guidelines.

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractors across multiple systems.
• Score the clinical reasoning quality of benchmark items and assess vignette accuracy and trap quality.
• Check whether the reasoning chain uses vignette facts to reach the answer and write short rationales.
• Work through three blind reads and physician adjudication to ensure high-quality AI evaluation.

Skills

Strong written English

Education

MD or DO with active US license

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. This opportunity involves evaluating medical AI benchmarks and providing structured feedback on clinical vignettes and reasoning.

We are seeking US-based physicians (MD/DO) with active licenses to participate part-time. Tasks are paid per accepted item, with potential rates up to $70 per hour depending on qualification tier and efficiency.