Clinical Project Lead (MD)

Lumos

San Francisco (CA)

On-site

USD 180,000 - 200,000

Full time

6 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Lumos, based in San Francisco, builds health AI evaluation tools to ensure clinical safety and accuracy. We are seeking a medical doctor to lead the clinical quality bar across data and physician networks.

This full-time role centers on creating expert cases, adjudicating peer work, and designing RL environments to test how medical models reason under real clinical conditions, collaborating with engineers and researchers.

Qualifications

  • MD or DO with US residency in internal medicine, family medicine, or cardiology.
  • US board certification in internal medicine, cardiovascular disease, or family medicine.
  • 3+ years post-residency clinical experience, with active practice.

Responsibilities

  • Build gold-standard clinical cases and reference solutions for frontier models.
  • Review and adjudicate work of physician experts; define quality standards.
  • Design and ship RL environments and verifiers for clinical agents.
  • Stress-test models to find hallucinations, red flags, and unsafe guidance; improve evaluations.
  • Be the credible clinical voice with customers and the physician network.

Skills

Clinical expertise
Evidence-based medicine
Clinical communication
Structured data & EHR concepts

Education

MD/DO from an accredited program
US residency in Internal Medicine, Family Medicine or Cardiology
US board certification in Internal Medicine / Cardiology / Family Medicine
3+ years of post-residency clinical experience

Job description

Lumos is a health and life-science AI evaluation company based in San Francisco. Our mission is to make health and life-science AI genuinely accurate and safe. We build the benchmarks, RL environments, and adversarial testing frameworks that frontier labs rely on to know whether their models can be trusted with science.

We’re growing our San Francisco team to support new product launches and foundational lab partnerships. You get startup speed and frontier-model stakes, working in the middle of the AI ecosystem.

The Role

We are looking for a medical doctor who will be at the center of how Lumos builds and evaluates clinical AI. This is a hands‑on, full‑time role at our San Francisco headquarters. The core of the work is:

  • Building gold‑standard clinical cases: the expert‑annotated tasks and reference solutions that become training and evaluation data for frontier models.
  • Reviewing and adjudicating the work of a network of physician experts, and owning the clinical quality bar everyone works to.
  • Designing and shipping RL environments with our research and engineering team. These are the worlds that test whether medical models and agents reason safely under real clinical conditions.

You’ll set the clinical quality criteria, find where models are weak or unsafe, and be the credible clinical voice for customers and our physician network.

What You Will Do
  • Build gold‑standard, expert‑annotated clinical cases and reference solutions across internal medicine, cardiology, and primary care. This covers diagnosis and differential reasoning, triage and risk stratification (including cardiovascular risk), chest pain and acute coronary syndrome workups, heart failure and arrhythmia management, ECG and cardiac test interpretation, longitudinal management, medication reconciliation, guideline‑concordant care (ACC/AHA, USPSTF, ADA, and others), and clinical documentation. All of it is grounded in realistic patient data (FHIR bundles, structured EHR) that becomes training and eval data for frontier models.
  • Review and adjudicate the work of a network of physician experts. Own the clinical quality bar, scoring rubrics, and inter‑rater reliability.
  • Design and ship RL environments and verifiers for clinical agents with our research and engineering team, turning what “correct” and “safe” mean into automated signal.
  • Stress‑test and red‑team models on clinical tasks. Find where they hallucinate, miss red flags, or give unsafe guidance, and turn those failures into lasting evaluations.
  • Be the credible clinical voice with customers (frontier labs and healthcare‑AI companies) and with the physician expert network you help recruit and calibrate.
What You Bring
  • MD or DO from an accredited program, with completed US residency in Internal Medicine, Family Medicine or Cardiology.
  • US board certification in one of the following:
  • Internal Medicine (ABIM or AOBIM)
  • Cardiovascular Disease (ABIM), following fellowship
  • Family Medicine (ABFM or AOBFP)
  • 3+ years of post‑residency (or post‑fellowship) clinical experience, with recent or active clinical practice.
  • A strong command of evidence‑based medicine, guideline‑concordant care, and clinical reasoning under uncertainty. You know the difference between an answer that sounds right and one that is clinically defensible.
  • Comfort working with structured clinical data and EHR concepts, and a willingness to work closely with engineers and researchers.
  • Rigor about correctness, reproducibility, and edge cases.
  • Excellent written and spoken communication.
Nice to Have
  • Hands‑on experience using AI tools in clinical practice. For example: ambient AI scribes and documentation assistants, LLM‑based clinical decision support, AI‑assisted ECG or imaging interpretation, EHR‑embedded AI features (such as inbox drafting or chart summarization), or general‑purpose LLMs used for clinical reasoning. You understand where these tools help, where they fail, and what that means for patient safety.
  • Prior experience annotating, evaluating, or red‑team­ing clinical AI or LLM systems.
  • Experience writing clinical guidelines, quality measures, or board‑style assessment content (for example ABIM, ABFM, or MKSAP‑style questions).
  • Publications in clinical medicine, evidence‑based medicine, cardiology, or medical AI.
  • Exposure to RL, verifier‑based scoring, or evaluation frameworks.
Work Authorization

Candidates must be authorized to work in the United States without sponsorship. Lumos does not sponsor work authorization for this role.

Compensation Range:

$180K - $200K

Lumos is an equal opportunity employer. We are committed to building a diverse and inclusive team and do not discriminate on the basis of race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, veteran status, or any other characteristic protected by applicable law. All qualified applicants will receive consideration for employment. We will consider qualified applicants with arrest and conviction records in line with the San Francisco Fair Chance Ordinance and the California Fair Chance Act.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Clinical Project Lead (MD)
Clinical Project Lead (MD)

Lumos AI Research • San Francisco (CA)

On-site
USD 180,000 - 300,000
Clinical Project Lead
Clinical Project Lead

Lumos AI Research • San Francisco (CA)

On-site
USD 180,000 - 240,000
Clinical AI Lead - MD, Medical AI Evaluation
Clinical AI Lead - MD, Medical AI Evaluation

Lumos AI Research • San Francisco (CA)

On-site
USD 180,000 - 300,000
Senior Clinical AI Project Lead
Senior Clinical AI Project Lead

Lumos AI Research • San Francisco (CA)

On-site
USD 180,000 - 240,000
Clinical AI Evaluation & Safety Lead
Clinical AI Evaluation & Safety Lead

Lumos • San Francisco (CA)

On-site
USD 180,000 - 200,000
Medical Director, Clinical AI, Product Development and Validation
Medical Director, Clinical AI, Product Development and Validation

Lumeris, Inc. • United States

Remote
USD 203,000 - 295,000
Medical, Vision, and Dental Plans
401k with company match
Paid Time Off (PTO) and Wellness Days
Strategy Lead
Strategy Lead

Luminai • San Mateo (CA)

On-site
USD 170,000 - 220,000
Director of Machine Learning (Healthcare AI)
Director of Machine Learning (Healthcare AI)

Nxt Level • United States

Hybrid
USD 150,000 - 200,000
Competitive salary
Meaningful equity
Direct line to CEO
+1
Strategy Lead
Strategy Lead

Luminai • San Francisco (CA)

On-site
USD 180,000 - 240,000
Sales Engineer (AI)
Sales Engineer (AI)

Luma Health • Salt Lake City (UT)

On-site
USD 120,000 - 140,000
Health benefits
Wellness programs
Stock options
+2