Healthcare AI Evaluation Scientist (Remote)

Appsierra Group; American Samoa; On-site; USD 200,000 - 250,000; Full time

Benefits offered by this job

Health insurance premium reimbursement

Paid time off

401(k) with company match

Equity compensation

Job summary

Appsierra Group is seeking a Member of Technical Staff, Medical Research to advance AI systems supporting healthcare, clinical and biomedical workflows. This remote, full-time role centers on evaluating medical AI with rigorous standards, benchmarks, and cross-disciplinary collaboration.

You will design evaluation frameworks, analyze model performance on clinical tasks, and develop datasets and environments to improve reliability and safety of healthcare AI systems.

Qualifications

• Advanced healthcare degree required (MD/DO/PhD/MPH/PharmD or related).
• Strong background in clinical practice, biomedical research, or public health.
• Excellent analytical reasoning and ability to communicate scientific findings.

Responsibilities

• Design and own evaluation frameworks for medical AI systems, including benchmarks and validation protocols.
• Conduct research on medical reasoning, evidence synthesis, and healthcare decision-making processes.
• Develop benchmark datasets and evaluation environments based on real-world clinical scenarios.
• Collaborate with researchers, clinicians, and engineers to evaluate and improve healthcare-focused AI systems.
• Analyze model performance, safety, reasoning quality, and failure patterns across medical use cases.
• Produce research reports, evaluation methodologies, and best-practice guidelines.
• Stay current with developments in medicine and AI and contribute to emerging standards.

Skills

Medical research

Analytical reasoning

Scientific communication

Cross-disciplinary collaboration

Education

MD/DO/PhD/MPH/PharmD or related

Job description

Appsierra Group is seeking a Member of Technical Staff, Medical Research to advance AI systems supporting healthcare, clinical and biomedical workflows. This remote, full-time role centers on evaluating medical AI with rigorous standards, benchmarks, and cross-disciplinary collaboration.

You will design evaluation frameworks, analyze model performance on clinical tasks, and develop datasets and environments to improve reliability and safety of healthcare AI systems.