Remote Medical Evaluation Architect for AI Benchmarks

YO AI Labs; Dallas (TX); Remote; USD 83,000 - 152,000; Part time

Job summary

YO AI Labs seeks Medical Evaluation Specialists, including medical students, residents, physicians, and biomedical professionals, to contribute clinical expertise for evaluating and improving next-generation AI systems.

You will create and validate challenging medical questions and answers to test advanced clinical reasoning, synthesize evidence, and document rationales with citations. This is a remote contractor role emphasizing high-quality medical content.

Qualifications

• Medical training is required.
• Strong research and source triangulation skills are essential.
• Exceptional attention to detail and written precision are expected.
• Demonstrated analytical thinking and clinical reasoning required.
• Experience with medical literature review and evidence synthesis preferred.
• Ability to work independently and meet deadlines in a remote setting.
• Experience with medical question writing, peer review, or medical education is a plus.

Responsibilities

• Create original, high-difficulty medical question-and-answer pairs across diagnosis, pathophysiology, pharmacology, guidelines, and clinical decision-making.
• Develop questions that require synthesis, interpretation, and genuine medical reasoning beyond recall.
• Research and verify answers using primary literature, guidelines, and authoritative references.
• Document rationale with citations and assess AI-generated responses for accuracy and evidence adherence.
• Refine questions to increase complexity while maintaining clinical validity.
• Review for ambiguity, errors, and inconsistencies to ensure clarity and defensibility.

Skills

Medical Training

Research & Source Triangulation

Attention to Detail

Written Precision

Analytical Thinking

Clinical Reasoning

Medical Literature Review

Evidence Synthesis

Self-Direction & Reliability

Medical Question Development

Job description

YO AI Labs seeks Medical Evaluation Specialists, including medical students, residents, physicians, and biomedical professionals, to contribute clinical expertise for evaluating and improving next-generation AI systems.

You will create and validate challenging medical questions and answers to test advanced clinical reasoning, synthesize evidence, and document rationales with citations. This is a remote contractor role emphasizing high-quality medical content.