Radiology AI Evaluation Specialist

Mpathic

United States

On-site

USD 165,000 - 276,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

mpathic.ai is seeking experienced radiologists to help evaluate the next generation of multimodal AI systems by assessing how they interpret medical images, radiology reports, and complex clinical instructions. This benchmark dataset project focuses on evaluating AI models for visual document understanding and instruction-following in the radiology domain.

You'll develop challenging, clinically realistic evaluation tasks, create expert reference answers, and identify where today's AI systems

Qualifications

  • MD, DO, or internationally equivalent medical degree.
  • Completion of an accredited Diagnostic Radiology residency (ACGME, AOA, or recognized international equivalent).
  • Current board certification in Diagnostic Radiology (ABR, FRCPC, FRANZCR, or recognized national equivalent).
  • Active, unrestricted medical license with no suspension, revocation, or ongoing disciplinary action.
  • At least 3 years of independent clinical practice following residency (training years excluded), recently within past two years.
  • Experience practicing in accredited or government-licensed healthcare settings (hospitals, academic medical centers, outpatient imaging centers, or private practice).

Responsibilities

  • Design clinically realistic evaluation tasks using radiology images, reports, and supporting clinical documentation.
  • Create expert-level prompts that assess visual document understanding, diagnostic reasoning, and instruction-following across radiology workflows.
  • Evaluate AI-generated responses using structured rubrics for diagnostic accuracy, completeness, safety, and adherence to instructions.
  • Identify challenging edge cases, subtle imaging findings, and common model failure modes across multiple imaging modalities.
  • Provide detailed written feedback that helps improve model performance and benchmark quality.
  • Collaborate directly with research teams at a leading AI lab to advance multimodal healthcare AI.
  • Participate in onboarding, calibration sessions, and ongoing quality reviews.

Skills

Unrestricted medical license
Board certification in radiology
Independent clinical practice 3+ years

Education

MD, DO, or internationally equivalent medical degree
Residency in Diagnostic Radiology (ACGME, AOA, or international equivalent)
Active board certification in Diagnostic Radiology (ABR, FRCPC, FRANZCR, or national equivalent)

Job description

mpathic.ai is seeking experienced radiologists to help evaluate the next generation of multimodal AI systems by assessing how they interpret medical images, radiology reports, and complex clinical instructions. This benchmark dataset project focuses on evaluating AI models for visual document understanding and instruction-following in the radiology domain.

You'll develop challenging, clinically realistic evaluation tasks, create expert reference answers, and identify where today's AI systems

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Radiology AI Benchmarking Specialist
Radiology AI Benchmarking Specialist

mpathic • United States

Remote
USD 138,000 - 276,000
Radiology Specialist (Contract)
Radiology Specialist (Contract)

Mpathic • United States

On-site
USD 165,000 - 276,000
Radiology Specialist (Contract)
Radiology Specialist (Contract)

mpathic • United States

Remote
USD 138,000 - 276,000
Remote Radiologist for AI Training & Evaluation
Remote Radiologist for AI Training & Evaluation

micro1 Inc. • United States

Remote
USD 110,000 - 220,000
Remote work
Part-time contractor
Remote Radiologist for AI Diagnostics & Validation
Remote Radiologist for AI Diagnostics & Validation

Crossing Hurdles • United States

Remote
CAD 69,000 - 110,000
Radiologist AI Trainer for Medical Imaging
Radiologist AI Trainer for Medical Imaging

DataAnnotation • Framingham (MA)

Remote
USD 83,000 - 172,000
Flexible, remote work
Remote AI Radiology Trainer — Cardiothoracic Imaging Expert
Remote AI Radiology Trainer — Cardiothoracic Imaging Expert

DataAnnotation • The Woodlands (TX)

Remote
USD 110,000 - 172,000
Remote Radiology AI Evaluation Expert (Part-Time)
Remote Radiology AI Evaluation Expert (Part-Time)

OpenTrain AI, Inc. • United States

Remote
USD 248,000 - 413,000
Remote AI-Training Radiologist for Medical Imaging
Remote AI-Training Radiologist for Medical Imaging

DataAnnotation • Newton (MA)

Remote
USD 172,000 - 358,000
Flexible remote
Project-based work
Medical Imaging AI Review Specialist
Medical Imaging AI Review Specialist

Globik AI • United States

Remote
USD 25,000 - 28,000