Part-Time Clinical AI Evaluator — Licensed Physician

Braintrust; Saint Paul (MN); Remote; USD 83,000 - 165,000; Part time

Job summary

Braintrust is seeking US-based, actively practicing physicians to participate in a Medical AI Evaluation Program focused on General and Internal Medicine. You will evaluate EHR vignettes with questions and distractors, score reasoning quality, and write concise rationales.

Work involves three independent blind reads with physician adjudication for final conclusions. This part-time, project-based opportunity requires 10-20 hours per week during active phases.

Qualifications

• MD or DO with active US medical license.
• Board certification or completed residency with recent direct patient care.
• Broad, multi-system diagnostic and treatment experience.
• Strong clinical reasoning and evidence-based practice.
• Experience in medical AI evaluation or annotation is a plus.
• Strong written English proficiency (C1+).

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractors.
• Score reasoning quality, vignette accuracy, and trap quality.
• Check whether reasoning reaches the correct answer from vignette facts.
• Write short rationales and participate in three independent blind reads with adjudication.

Skills

MD/DO

US medical license

Board certification or residency

Clinical reasoning

Medical AI evaluation

English writing

Education

Medical degree (MD/DO)

Job description

Braintrust is seeking US-based, actively practicing physicians to participate in a Medical AI Evaluation Program focused on General and Internal Medicine. You will evaluate EHR vignettes with questions and distractors, score reasoning quality, and write concise rationales.

Work involves three independent blind reads with physician adjudication for final conclusions. This part-time, project-based opportunity requires 10-20 hours per week during active phases.