Founding Medical AI Researcher

Discover International; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Discover International, a confidential, early-stage AI company, seeks a Founding Lead Researcher to build its medical AI evaluation function from the ground up.

This high-autonomy role lets you define how advanced AI systems are tested for accuracy, reliability, robustness, and real-world usefulness.

Qualifications

• Strong experience in ML/AI evaluation, benchmarking, validation, or experimental research.
• Excellent statistics and experimental-design skills.
• Strong Python/software-engineering ability.
• Hands-on experience with LLMs, multimodal AI, agents, computer vision, or related systems.
• Strong scientific judgement and technical writing.

Responsibilities

• Build evaluation and benchmarking methodologies for modern AI systems.
• Design datasets, experiments, annotation frameworks, and validation processes.
• Develop rigorous approaches to measuring factuality, evidence use, reasoning, uncertainty, calibration, and robustness.
• Build reproducible evaluation infrastructure in Python.
• Work closely with ML researchers and engineers to turn findings into model improvements.
• Identify weaknesses and failure modes in existing evaluation approaches.
• Establish scientific standards for how model performance is measured and reported.

Skills

ML evaluation

Benchmarking

Experimental design

Python

Statistics

Scientific writing

LLMs

Multimodal AI

Tools

Python

PyTorch

TensorFlow

Job description

Remote | Full-time | Founding Research Role

We’re representing a confidential, early-stage AI company seeking a Founding Lead Researcher to build its medical AI evaluation function from the ground up.

This is a high-autonomy role for someone who wants to define how advanced AI systems are tested for accuracy, reliability, robustness, and real-world usefulness.

The Role:

• Build evaluation and benchmarking methodologies for modern AI systems.
• Design datasets, experiments, annotation frameworks, and validation processes.
• Develop rigorous approaches to measuring factuality, evidence use, reasoning, uncertainty, calibration, and robustness.
• Build reproducible evaluation infrastructure in Python.
• Work closely with ML researchers and engineers to turn findings into model improvements.
• Identify weaknesses and failure modes in existing evaluation approaches.
• Establish scientific standards for how model performance is measured and reported.
• Strong experience in ML/AI evaluation, benchmarking, validation, or experimental research.
• Excellent statistics and experimental-design skills.
• Strong Python/software-engineering ability.
• Hands-on experience with LLMs, multimodal AI, agents, computer vision, or related systems.
• Strong scientific judgement and technical writing.
• Ability to operate independently in an early-stage environment.

Experience in biomedical AI, healthcare, medical imaging, information retrieval, human evaluation, or evidence-grounded AI would be particularly relevant.

Why This Role?

You’ll be joining early enough to define the evaluation function rather than inherit one. The role offers significant autonomy, technical ownership, and the opportunity to influence how sophisticated AI systems are assessed.

Competitive salary + equity + benefits.

The client’s identity and further details will be shared with shortlisted candidates during the recruitment process.