Remote Medical Evaluation Architect for AI Testing

YO AI Labs; Palo Alto (CA); Remote; USD 83,000 - 165,000; Full time

Job summary

YO AI Labs is seeking Medical Evaluation Specialists to craft and validate challenging medical questions and answers for AI evaluation. Remote contractor role focusing on clinical expertise, literature review, and rigorous reasoning.

You will develop high-difficulty items spanning diagnosis, pathophysiology, pharmacology, and guidelines, ensuring accuracy and defensible explanations for AI benchmarks.

Qualifications

• Medical training or equivalent biomedical expertise.
• Ability to locate, interpret, and synthesize primary literature and guidelines.
• Strong written English with high attention to detail.
• Experience developing clinical questions with rigorous vetting.

Responsibilities

• Create original high-difficulty medical Q&A pairs across diagnosis, pathophysiology, pharmacology, and guidelines.
• Develop questions requiring synthesis and clinical reasoning beyond recall.
• Research answers using primary literature and clinical guidelines; cite sources.
• Document rationale and support with citations.
• Review questions for ambiguity and factual accuracy.

Skills

Medical Training

Research & Source Triangulation

Attention to Detail

Written Precision

Analytical Thinking

Clinical Reasoning

Medical Literature Review

Evidence Synthesis

Self-Direction & Reliability

Medical Question Development

Job description

YO AI Labs is seeking Medical Evaluation Specialists to craft and validate challenging medical questions and answers for AI evaluation. Remote contractor role focusing on clinical expertise, literature review, and rigorous reasoning.

You will develop high-difficulty items spanning diagnosis, pathophysiology, pharmacology, and guidelines, ensuring accuracy and defensible explanations for AI benchmarks.