Clinical Project Lead (MD)

Lumos AI Research

San Francisco (CA)

On-site

USD 180,000 - 300,000

Full time

7 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Lumos AI Research in San Francisco seeks a medical doctor to lead clinical evaluation efforts for medical AI. This full-time role centers on creating gold-standard clinical cases, adjudicating expert work, and collaborating with engineers to build RL environments and verifiers that test model safety in real clinical scenarios.

You will drive the clinical quality criteria, identify model weaknesses, and serve as the credible clinical voice for customers and the physician network.

Qualifications

  • MD or DO with US residency in Internal Medicine, Family Medicine, or Cardiology.
  • US board certification in Internal Medicine, Cardiovascular Disease, or Family Medicine.
  • 3+ years of post-residency clinical experience.
  • Strong command of evidence-based medicine and clinical reasoning under uncertainty.
  • Comfort working with structured clinical data and EHR concepts, and collaboration with engineers/researchers.

Responsibilities

  • Build gold-standard, expert-annotated clinical cases and reference solutions across internal medicine, cardiology, and primary care.
  • Review and adjudicate the work of a network of physician experts; own the clinical quality bar and rubrics.
  • Design and ship RL environments and verifiers for clinical agents with the research/engineering team; translate correctness and safety into signals.
  • Stress-test and red-team models on clinical tasks; identify red flags and unsafe guidance to improve evaluations.
  • Be the credible clinical voice with customers and with the physician network you recruit and calibrate.

Skills

Clinical reasoning
Evidence-based medicine
Communication skills
EHR data handling
Medical knowledge

Education

MD/DO with US residency in Internal Medicine / Family Medicine / Cardiology

Job description

About Lumos

Lumos is a health and life-science AI evaluation company based in San Francisco. Our mission is to make health and life-science AI genuinely accurate and safe. We build the benchmarks, RL environments, and adversarial testing frameworks that frontier labs rely on to know whether their models can be trusted with science.

We’re growing our San Francisco team to support new product launches and foundational lab partnerships. You get startup speed and frontier-model stakes, working in the middle of the AI ecosystem.

The Role

We are looking for a medical doctor who will be at the center of how Lumos builds and evaluates clinical AI. This is a hands-on, full-time role at our San Francisco headquarters. The core of the work is:

  • Building gold-standard clinical cases: the expert-annotated tasks and reference solutions that become training and evaluation data for frontier models.

  • Reviewing and adjudicating the work of a network of physician experts, and owning the clinical quality bar everyone works to.

  • Designing and shipping RL environments with our research and engineering team. These are the worlds that test whether medical models and agents reason safely under real clinical conditions.

You’ll set the clinical quality criteria, find where models are weak or unsafe, and be the credible clinical voice for customers and our physician network.

What You Will Do
  • Build gold-standard, expert-annotated clinical cases and reference solutions across internal medicine, cardiology, and primary care. This covers diagnosis and differential reasoning, triage and risk stratification (including cardiovascular risk), chest pain and acute coronary syndrome workups, heart failure and arrhythmia management, ECG and cardiac test interpretation, longitudinal management, medication reconciliation, guideline-concordant care (ACC/AHA, USPSTF, ADA, and others), and clinical documentation. All of it is grounded in realistic patient data (FHIR bundles, structured EHR) that becomes training and eval data for frontier models.

  • Review and adjudicate the work of a network of physician experts. Own the clinical quality bar, scoring rubrics, and inter-rater reliability.

  • Design and ship RL environments and verifiers for clinical agents with our research and engineering team, turning what "correct" and "safe" mean into automated signal.

  • Stress-test and red-team models on clinical tasks. Find where they hallucinate, miss red flags, or give unsafe guidance, and turn those failures into lasting evaluations.

  • Be the credible clinical voice with customers (frontier labs and healthcare-AI companies) and with the physician expert network you help recruit and calibrate.

What You Bring
  • MD or DO from an accredited program, with completed US residency in Internal Medicine, Family Medicine or Cardiology.

  • US board certification in one of the following:

    • Internal Medicine (ABIM or AOBIM)
    • Cardiovascular Disease (ABIM), following fellowship
    • Family Medicine (ABFM or AOBFP)
  • 3+ years of post-residency (or post-fellowship) clinical experience, with recent or active clinical practice.

  • A strong command of evidence-based medicine, guideline-concordant care, and clinical reasoning under uncertainty. You know the difference between an answer that sounds right and one that is clinically defensible.

  • Comfort working with structured clinical data and EHR concepts, and a willingness to work closely with engineers and researchers.

  • Rigor about correctness, reproducibility, and edge cases.

  • Excellent written and spoken communication.

Nice to Have
  • Hands-on experience using AI tools in clinical practice. For example: ambient AI scribes and documentation assistants, LLM-based clinical decision support, AI-assisted ECG or imaging interpretation, EHR-embedded AI features (such as inbox drafting or chart summarization), or general-purpose LLMs used for clinical reasoning. You understand where these tools help, where they fail, and what that means for patient safety.

  • Prior experience annotating, evaluating, or red-teaming clinical AI or LLM systems.

  • Experience writing clinical guidelines, quality measures, or board-style assessment content (for example ABIM, ABFM, or MKSAP-style questions).

  • Publications in clinical medicine, evidence-based medicine, cardiology, or medical AI.

  • Exposure to RL, verifier-based scoring, or evaluation frameworks.

Work Authorization

Candidates must be authorized to work in the United States without sponsorship. Lumos does not sponsor work authorization for this role.

Start date: Flexible, ASAP preferred

Equal Opportunity

Lumos is an equal opportunity employer. We are committed to building a diverse and inclusive team and do not discriminate on the basis of race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, veteran status, or any other characteristic protected by applicable law. All qualified applicants will receive consideration for employment. We will consider qualified applicants with arrest and conviction records in line with the San Francisco Fair Chance Ordinance and the California Fair Chance Act.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Clinical Project Lead
Clinical Project Lead

Lumos AI Research • San Francisco (CA)

On-site
USD 180,000 - 240,000
Clinical Project Lead (MD)
Clinical Project Lead (MD)

Lumos • San Francisco (CA)

On-site
USD 180,000 - 200,000
Clinical AI Lead - MD, Medical AI Evaluation
Clinical AI Lead - MD, Medical AI Evaluation

Lumos AI Research • San Francisco (CA)

On-site
USD 180,000 - 300,000
Clinical AI Evaluation & Safety Lead
Clinical AI Evaluation & Safety Lead

Lumos • San Francisco (CA)

On-site
USD 180,000 - 200,000
Senior Clinical AI Project Lead
Senior Clinical AI Project Lead

Lumos AI Research • San Francisco (CA)

On-site
USD 180,000 - 240,000
Medical Director, Clinical AI, Product Development and Validation
Medical Director, Clinical AI, Product Development and Validation

Lumeris, Inc. • United States

Remote
USD 203,000 - 295,000
Medical, Vision, and Dental Plans
401k with company match
Paid Time Off (PTO) and Wellness Days
Director of Machine Learning (Healthcare AI)
Director of Machine Learning (Healthcare AI)

Nxt Level • United States

Hybrid
USD 150,000 - 200,000
Competitive salary
Meaningful equity
Direct line to CEO
+1
Senior Data Scientist – LLM / Clinical NLP
Senior Data Scientist – LLM / Clinical NLP

Vecten • United States

Remote
USD 140,000 - 190,000
Cursor Pro licenses
Claude Pro licenses
Remote-first autonomy
Strategy Lead
Strategy Lead

Luminai • San Mateo (CA)

On-site
USD 170,000 - 220,000
Senior Applied Scientist, Reinforcement Learning
Senior Applied Scientist, Reinforcement Learning

Hippocratic AI Inc. • Menlo Park (CA)

On-site
USD 180,000 - 240,000