Clinical AI Evaluation Specialist

mpathic; United States; Remote; USD 83,000 - 165,000; Part time

Job summary

mpathic, a regulatory science company, is recruiting experienced ML and data science professionals to serve as technical reviewers on expert evaluation panels for healthcare AI deployments in real-world clinical settings. You will independently assess evidence, validate methodologies, and communicate actionable recommendations to technical and non-technical stakeholders.

This is an evaluation and scientific review role, not a model development position.

Qualifications

• At least 3 years of professional or applied research in ML, data science, biostatistics, or related field with evaluating/validating predictive models or AI systems.
• Demonstrated understanding of model evaluation methodologies, metrics, validation design, and bias sources.
• Experience assessing model performance on independent/held-out data and generalizability.
• Hands-on experience evaluating healthcare AI systems (clinical NLP, imaging, tabular models, or LLMs).
• Ability to critically assess technical research and identify limitations.
• Strong analytical writing; ability to communicate to non-technical audiences.
• Ability to work independently within multidisciplinary teams.
• Willingness to sign confidentiality and conflict-of-interest disclosures.

Responsibilities

• Evaluate model performance and validation methodologies, data sources, and uncertainty.
• Assess data quality and representativeness for clinical applications across populations and settings.
• Evaluate technical reliability and robustness under atypical inputs.
• Review evaluation plans with defensible methods, benchmarks, and success criteria.
• Assess proposed safeguards and human-in-the-loop processes for feasibility.
• Evaluate monitoring plans for degradation, drift, and change management.
• Collaborate with clinicians and technical experts to identify cross-disciplinary concerns.
• Produce structured findings with strengths, limitations, and actionable recommendations.
• Participate in reviewer calibration and quality assurance.
• Maintain independence and confidentiality in reviewing materials.

Skills

ML evaluation

Biostatistics

Clinical informatics

Independent work

Analytical writing

Regulatory awareness

Confidentiality

Education

Master's or PhD in ML/CS/Bioinformatics

Clinical informatics background

Tools

TRIPOD+AI

STARD-AI

CONSORT-AI

DECIDE-AI

Model cards

Job description

mpathic, a regulatory science company, is recruiting experienced ML and data science professionals to serve as technical reviewers on expert evaluation panels for healthcare AI deployments in real-world clinical settings. You will independently assess evidence, validate methodologies, and communicate actionable recommendations to technical and non-technical stakeholders.

This is an evaluation and scientific review role, not a model development position.