Part-Time Medical AI Evaluation Physician (Up to $70/hr)

Mindrift; Eagle (ID); Remote; USD 80,000 - 113,000; Part time

Job summary

Mindrift invites US-based physicians to participate in a project-based AI evaluation program focused on testing, evaluating, and improving clinical AI systems. You will review EHR vignettes, score reasoning quality, and write concise rationales.

The work is part-time, non-permanent, and paid per accepted task, with rates up to $70/hour. Ideal candidates hold an MD/DO license, board certification or residency, and strong English writing skills.

Qualifications

• MD or DO degree or equivalent medical degree.
• Active US medical license (verified).
• Board certification or completed residency with recent direct patient care.
• Broad diagnostic/treatment experience across internal, family, emergency, or hospital medicine.
• Strong clinical reasoning and application of evidence-based guidelines.
• Experience in medical AI evaluation or medical annotation/QA is a plus.
• Strong written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes with questions, proposed answers, and distractors across multiple systems.
• Score the clinical reasoning quality of items: vignette accuracy, completeness, and trap quality.
• Check whether reasoning chains reach the answer from vignette facts and write short rationales.
• Work within three blind reads followed by physician adjudication to ensure accuracy.
• Contribute to high-quality medical AI evaluation used to improve AI tools.

Education

MD or DO degree

Active US medical license

Board certification or completed residency

Broad multi-system diagnostic and treatment experience

Strong clinical reasoning

Strong written English (C1+)

Job description

Mindrift invites US-based physicians to participate in a project-based AI evaluation program focused on testing, evaluating, and improving clinical AI systems. You will review EHR vignettes, score reasoning quality, and write concise rationales.

The work is part-time, non-permanent, and paid per accepted task, with rates up to $70/hour. Ideal candidates hold an MD/DO license, board certification or residency, and strong English writing skills.