Part-Time Physician - AI Clinical Vignettes Evaluator

Braintrust; Boise (ID); Remote; USD 165,000 - 248,000; Part time

Job summary

Braintrust seeks US-based physicians (MD/DO) to evaluate clinical EHR vignettes for an AI evaluation program. Focus areas include General and Internal Medicine as well as multi-system diagnosis and treatment tasks.

You will assess reasoning quality, provide concise rationales, and participate in blinded reads and adjudication. Requirements include an active US medical license, board certification or residency with direct patient care, and strong written English.

Qualifications

• MD or DO degree required.
• Active, unrestricted US medical license required.
• Board certification or completed residency with direct patient care.
• Broad, multi-system diagnostic and treatment experience.
• Strong clinical reasoning and application of guidelines.
• Experience in medical AI evaluation or annotation is a plus.
• Excellent written English required (C1+).

Responsibilities

• Evaluate clinical EHR vignettes linked to questions, proposed answers, and distractors.
• Score clinical reasoning quality, vignette accuracy, completeness, and trap quality.
• Assess whether reasoning leads from vignette facts to the correct answer; write concise rationales.
• Participate in three blind reads and physician adjudication.

Skills

Clinical reasoning

Written English (C1+)

Multi-system diagnostic experience

Medical AI evaluation experience

Education

MD/DO degree

Active US medical license

Board certification or residency with direct patient care

Job description

Braintrust seeks US-based physicians (MD/DO) to evaluate clinical EHR vignettes for an AI evaluation program. Focus areas include General and Internal Medicine as well as multi-system diagnosis and treatment tasks.

You will assess reasoning quality, provide concise rationales, and participate in blinded reads and adjudication. Requirements include an active US medical license, board certification or residency with direct patient care, and strong written English.