ML Researcher, Medical Evaluation

VitalSyncLabs; Madison, Northern (WI, KY); Hybrid; USD 40,000 - 66,000; Part time

Job summary

VitalSyncLabs seeks a part-time ML Researcher with medical or biomedical expertise to evaluate how accurately models interpret and present health information. You’ll assess factual accuracy, context preservation, and uncertainty handling, identifying safety-related failures and guiding improvement in collaboration with engineers and clinical reviewers.

The role surveys medical literature, designs human-review studies, and develops criteria for model evaluation.

Qualifications

• Background in medicine, biomedical science, clinical informatics, or related research.
• Ability to evaluate medical literature and distinguish established evidence from uncertainty.
• Familiarity with research methods, statistics, and ML evaluation limitations.
• Ability to write precise review criteria and explain medical considerations to technical colleagues.
• Comfort with structured datasets and model outputs.

Responsibilities

• Assess factual accuracy, completeness, temporal reasoning, and handling of missing or contradictory information.
• Identify misleading interpretations, fabricated details, inappropriate reassurance, and other safety-relevant failure modes.
• Review medical literature and translate relevant evidence into clear expectations that engineers can test.
• Design structured comparisons and human-review studies, documenting disagreements and limitations.
• Work with engineering contributors and qualified clinical reviewers to turn findings into improvements and identify issues requiring further review.
• Medically grounded evaluation cases and review criteria for model outputs and multi-step interactions.

Skills

Medical background

Clinical informatics

Literature evaluation

Python

R

Tools

Python

R

Job description

# ML Researcher, Medical EvaluationMadison, Wisconsin · Remote within the United StatesPart-time · EmployeeWe’re looking for a researcher with medical or biomedical expertise to help assess how accurately VitalSync’s models interpret and communicate health information.Apply for this role## About the roleThe work goes beyond checking whether an answer sounds plausible. You’ll examine whether models preserve the meaning of the available evidence, account for relevant context, recognize uncertainty, and avoid unsupported medical conclusions. Your findings will help shape evaluation methods, model improvements, and product decisions.This is a research and product-evaluation role, not a patient-care position.VitalSyncLabs is building a health-context platform to help people understand what happens between medical visits. Our work connects personal experiences, health measurements, and history over time while preserving where information came from and what remains uncertain.We’re an early-stage company based in Madison, Wisconsin. We bring together software, machine learning, and medical knowledge to build tools that support people and clinicians in understanding health—not replace their judgment.Apply with your résumé. You may also include a GitHub profile, portfolio, LinkedIn profile, publications, or other relevant work. Please share only material you have permission to disclose; do not include patient information or confidential employer research.## What you’ll do* Assess factual accuracy, completeness, temporal reasoning, and the handling of missing or contradictory information.* Identify misleading interpretations, fabricated details, inappropriate reassurance, and other safety-relevant failure modes.* Review medical literature and translate relevant evidence into clear expectations that engineers can test.* Design structured comparisons and human-review studies, documenting disagreements and limitations.* Work with engineering contributors and qualified clinical reviewers to turn findings into improvements and identify issues requiring further review.* Medically grounded evaluation cases and review criteria for model outputs and multi-step interactions.## What we’re looking for* A background in medicine, biomedical science, clinical informatics, or a related research discipline. Clinical licensure is not required.* Strong skills in evaluating medical literature and distinguishing established evidence from uncertainty.* Familiarity with research methods, statistics, and the limitations of machine-learning evaluation.* The ability to write precise review criteria and explain medical considerations to technical colleagues.* Comfort working with structured datasets and model outputs; Python or R experience is helpful.### Nice to have* Experience in clinical practice, health-AI evaluation, biomedical NLP, or medical annotation would be valuable.* We’re interested in thoughtful, evidence-based judgment as well as technical ability.## How we hireWe review applications for relevant experience, skills, and interest in what we’re building. If we move forward, we’ll arrange a conversation about your background, the role, and working at VitalSyncLabs. Depending on the position, we may discuss previous projects, research, or how you approach relevant problems. You’ll have time to ask questions, and we’ll explain any additional steps in advance.Sponsorship is not available for this role.