Healthcare AI Evaluation Scientist

Appsierra Group; United States; On-site; USD 200,000 - 250,000; Full time

Benefits offered by this job

Health insurance reimbursement

401(k) match

Paid time off

Job summary

Appsierra Group is seeking a Member of Technical Staff, Medical Research to advance AI systems for healthcare, clinical, and biomedical workflows. The role emphasizes evaluating medical AI across reasoning, evidence synthesis, and decision support.

You'll develop benchmarks, datasets, and evaluation standards while collaborating across research, clinical, and technical teams to ensure reliable healthcare AI performance.

Qualifications

• Advanced degree and expertise in medicine or related healthcare field.
• Strong background in medical research and clinical practice.
• Experience designing research studies and evaluating evidence.
• Excellent analytical reasoning and scientific communication skills.

Responsibilities

• Design and own evaluation frameworks for healthcare AI systems, including benchmarks and validation protocols.
• Conduct research into medical reasoning, evidence-based decision-making, and healthcare information synthesis.
• Develop benchmark datasets and evaluation environments from real-world healthcare scenarios.
• Collaborate with researchers, clinicians, and engineers to evaluate and improve healthcare-focused AI systems.
• Analyze model performance, safety, reasoning quality, and failure patterns across medical use cases.
• Produce research reports, evaluation methodologies, and best-practice guidelines.

Skills

Medical research

Clinical practice

Research design

Evidence evaluation

Analytical reasoning

Education

MD/DO/PhD/MPH/PharmD

Job description

Appsierra Group is seeking a Member of Technical Staff, Medical Research to advance AI systems for healthcare, clinical, and biomedical workflows. The role emphasizes evaluating medical AI across reasoning, evidence synthesis, and decision support.

You'll develop benchmarks, datasets, and evaluation standards while collaborating across research, clinical, and technical teams to ensure reliable healthcare AI performance.