Physician AI QA Evaluator — US‑Based, Part‑Time

Braintrust; Jefferson City (MO); On-site; USD 83,000 - 152,000; Part time

Job summary

Mindrift is seeking US-based physicians (MD/DO) to participate in a medical AI evaluation program that tests clinical EHR vignettes. Focus areas include internal and general medicine, with tasks that require careful evaluation of diagnoses, treatments, and reasoning.

Contributors will score reasoning quality, assess vignette accuracy, and write concise rationales. Tasks are part-time, non-permanent and suitable for flexible weekly hours.

Qualifications

• MD or DO degree and an active, unrestricted US medical license.
• Board certification or completed residency training with recent direct patient care.
• Broad, multi-system diagnostic and treatment experience (internal, family, emergency, or hospital medicine).
• Strong clinical reasoning: differential diagnosis, next-step management, and application of evidence-based guidelines.

Responsibilities

• Evaluate clinical EHR vignettes with questions, proposed answers, and distractors across multiple systems.
• Score the clinical reasoning quality, vignette accuracy, and trap quality.
• Check whether reasoning traces reach the answer from vignette facts and write short rationales.
• Work within three independent blind reads followed by physician adjudication.

Skills

Strong clinical reasoning

Written English (C1+)

Medical knowledge across multiple-rows

Vignette evaluation/QA experience

Education

MD or DO

Active US medical license

Board certification or residency training

Job description

Mindrift is seeking US-based physicians (MD/DO) to participate in a medical AI evaluation program that tests clinical EHR vignettes. Focus areas include internal and general medicine, with tasks that require careful evaluation of diagnoses, treatments, and reasoning.

Contributors will score reasoning quality, assess vignette accuracy, and write concise rationales. Tasks are part-time, non-permanent and suitable for flexible weekly hours.