Physician: AI Clinical Vignettes Evaluator (Part-Time, US-Based)

Braintrust; Harrisburg (Dauphin County); On-site; USD 138,000 - 207,000; Part time

Job summary

Braintrust is seeking US-based physicians (MD/DO) to participate in a project-based medical AI evaluation program. You will evaluate clinical EHR vignettes across multiple systems, score reasoning quality, and provide brief rationales. This role emphasizes strong clinical judgment, English proficiency, and experience in medical annotation.

Part-time, non-permanent project. Contributors will work through blind reads and physician adjudication, with a focus on accuracy and safety in assessment.

Qualifications

• MD or DO degree required with active US medical license
• Board certification or completed residency training
• Broad multi-system diagnostic and treatment experience
• Strong clinical reasoning and application of guidelines
• Experience in medical AI evaluation or annotation is a plus
• Excellent written English (C1+)

Responsibilities

• Evaluate clinical EHR vignettes with questions, proposed answers, and distractors across multiple systems
• Score reasoning quality: accuracy, completeness, deduction, and trap quality
• Assess whether reasoning traces reach the correct conclusion from vignette facts
• Write concise rationales; participate in adjudication rounds as part of a blind review process

Skills

Strong English proficiency

Clinical reasoning

Attention to detail

Medical annotation experience

Education

MD/DO degree

US medical license (active)

Board certification or residency training

Job description

Braintrust is seeking US-based physicians (MD/DO) to participate in a project-based medical AI evaluation program. You will evaluate clinical EHR vignettes across multiple systems, score reasoning quality, and provide brief rationales. This role emphasizes strong clinical judgment, English proficiency, and experience in medical annotation.

Part-time, non-permanent project. Contributors will work through blind reads and physician adjudication, with a focus on accuracy and safety in assessment.