Radiology AI Benchmarking Specialist

mpathic; United States; Remote; USD 138,000 - 276,000; Part time

Job summary

mpathic.ai is seeking experienced radiologists to evaluate next-gen multimodal AI systems in radiology. You will design tasks evaluating how models interpret images, reports, and clinical instructions, crafting expert prompts and rubrics to judge diagnostic accuracy, safety, and adherence to instructions.

You will collaborate with leading AI researchers, participate in onboarding and calibration, and help identify edge cases across imaging modalities to improve benchmark quality.

Qualifications

• MD/DO or internationally equivalent medical degree.
• Completion of an accredited Diagnostic Radiology residency.
• Current board certification in Diagnostic Radiology.
• Active, unrestricted medical license with no suspensions or ongoing disciplinary actions.
• At least 3 years of independent clinical practice after residency.

Responsibilities

• Design clinically realistic evaluation tasks using radiology images, reports, and clinical documentation.
• Create expert-level prompts that assess visual document understanding and radiology workflows.
• Evaluate AI-generated responses using rubrics for diagnostic accuracy, completeness, safety, and adherence to instructions.
• Identify challenging edge cases and model failure modes across imaging modalities.
• Provide detailed written feedback to improve model performance and benchmark quality.
• Collaborate with research teams to advance multimodal healthcare AI.
• Participate in onboarding, calibration sessions, and quality reviews.

Skills

Radiology experience

Independent clinical practice

Active medical license

Education

MD/DO or internationally equivalent medical degree

Completed Diagnostic Radiology residency

Board certification in Diagnostic Radiology

Job description

mpathic.ai is seeking experienced radiologists to evaluate next-gen multimodal AI systems in radiology. You will design tasks evaluating how models interpret images, reports, and clinical instructions, crafting expert prompts and rubrics to judge diagnostic accuracy, safety, and adherence to instructions.

You will collaborate with leading AI researchers, participate in onboarding and calibration, and help identify edge cases across imaging modalities to improve benchmark quality.