Clinical AI Engineer

Remote Worker LTD.; United States; Remote; USD 150,000 - 210,000; Full time

Job summary

Heidi is seeking a doctor who has moved into AI to join Clinical AI R&D as a Clinical AI Engineer. You’ll define evaluation plans, run studies to journal standards, and monitor product performance after launch.

You’ll build LLM judges and rubrics, analyze results, and write up findings for regulatory and academic use. You’ll also implement changes to improve clinical safety and performance.

Qualifications

• Medical degree with independent clinical judgment.
• Academic depth in study design, statistics and reliability measures.
• Experience evaluating medical software or clinical AI, including LLM-based judging pipelines.
• Engineering capability to read and modify production code (Python/TypeScript).
• Ability to work independently on long-running projects and communicate clearly.

Responsibilities

• Design evaluations for greenfield products to academic standards.
• Build, validate and calibrate LLM-based judges and rubrics.
• Run evaluations, perform statistics, and document results for regulatory/academic use.
• Monitor post-market output quality and respond to feedback with data.
• Make code and rubric updates based on evidence.
• Publish findings in peer-reviewed venues.

Skills

Medical degree

Independent clinical judgment

Academic writing

Production code familiarity

Education

MD or DO

Tools

Python

TypeScript

Job description

We’re Heidi.

We're building the future of healthcare by giving every clinician the earth's finest AI Care Partner. Our platform has absorbed the administrative chaos of 175 million patient visits and supported 67 million clinical hours. Today, we support 2.8 million patient sessions a week across 190+ countries, in 110 languages and over 200 specialties.

Healthcare systems are failing us; clinicians spend more time on documentation than on patients, and the human connection that makes medicine worth practicing is eroding. Our mission is simple: double the world’s capacity for care and strengthen the human connection at its heart.

We found product-market fit with a freemium medical scribe that clinicians love. Now, we're expanding. Every task a clinician hands to Heidi is a patient who feels more attended to, a health system unclogged, and a clinician who gets to be a clinician again.

We’ve grown annual recurring revenue from $1 million to $50 million in two years.

To go further, we’ve secured US$340 million: a $100 million Series C led by Blackbird, with Phoenix Court, Point72 Private Investments and Headline, alongside a $240 million growth investment led by General Catalyst’s Customer Value Fund.

If you want to join us in doing work worth shipping, jump in.

The role

We’re looking for a doctor who has moved into AI and can code, to join Clinical AI R&D as a Clinical AI Engineer. You’ll sit inside a product pod and own the evidence for its product: define the evaluation before anyone builds, run it to a standard a journal reviewer would accept, keep watching the product once it ships, publish what the evidence shows, and change the product when the evidence says to.

The pod’s engineers can change the product but can’t judge whether a discharge summary missed something a consultant would catch. Clinicians can make that call but can’t open a pull request. You do both, and you treat the gap between a model’s output and good clinical practice as the thing you’re here to measure and close.

The loop doesn’t end at launch. You stay with the product for its life in market.

What you’ll do

• Design the evaluation. Define evaluations for greenfield products to an academic standard: endpoints, comparators, ground truth, sample size, analysis plan. Set the bar before the result arrives.

• Build the measurement. Build, optimize and validate the LLM judges and rubrics that score the product at scale. Report their agreement with clinician ground truth, not their convenience, and retire the ones that drift.

• Run it and write it up. Execute the evaluation, do the statistics, and write it up once in a form that serves the pod, the regulatory file and an academic submission.

• Run post-market surveillance. Monitor output quality in production and respond to regulatory feedback with data. Our clinical safety team triages complaints and incidents; you’re their counterpart in the pod, working out what their findings mean for the product and what should change.

• Fix what the evidence finds. Prompt and rubric changes, evaluation harness changes, code changes in the pod’s codebase, shipped through the pod’s normal review and release process.

• Publish. Contribute to peer-reviewed literature on the translational benefit of the product.

• Be the clinician in the room. Catch the output that reads well and is wrong. Hold the line on clinical nuance when a shortcut would ship faster.

What you'll need

• A medical degree and clinical practice with independent clinical judgment. This role runs on medical nuance, and this requirement doesn’t flex.

• Academic depth. Study design, inferential statistics, agreement and reliability measures, sample size, handling of multiple comparisons. Thesis or publication-level work, not a methods module.

• Experience evaluating medical software or clinical AI, including building or validating LLM-as-judge pipelines.

• Enough engineering to work in a production codebase. Read it, understand what a change will do, make the change in Python or TypeScript, and get it through review. Prompt engineering and AI-assisted coding are daily tools here, and they’re the floor, not the ceiling.

• Independence. You can run a project for weeks with light direction, and you say early when it’s stuck.

• Clear writing. Evaluation reports, surveillance summaries and manuscripts are the primary output of this role.

Nice to have

• Post-market surveillance or vigilance work under a medical device framework (TGA, MHRA, FDA or EU MDR). Appetite to learn counts for more than existing depth.

• Peer-reviewed publications on clinical AI or LLM evaluation.

• Machine learning background beyond prompting, such as fine-tuning or classical model evaluation.

• Specialty depth in an area Heidi is moving into.

• An academic or clinical-site network able to host a study.

How we show up

• Build for the next decade, not next quarter. Our targets are outrageous on purpose. The world's health doesn't have the luxury of incrementalism.

• Lead, don't wait. We treat tomorrow's problems today. Sometimes we build what's needed before it's wanted, and we're fine with that.

• Follow the evidence. Trust the patient. We pursue truth relentlessly. But when the subjective and objective disagree, we treat the patient, not the numbers. Ego is a comorbidity we can't afford.

• Own the outcome. Everyone here carries the company. Raise problems with solutions, solve them end-to-end, and never be a bystander.

• Ship, measure, go again. A button today, a workflow tomorrow. More iterations beat better planning. We're precise at pace, not reckless.

• Live in clinicians' reality. Not the ideal workflow, the twenty-patients-before-lunch actual one. We build for exhausted humans, and we'd better be decent ones while we do it.

Why Heidi?

You’ll join a team measured on real-world impact, not press cycles. We live and breathe the challenges of modern health systems, and are laser-focused on exacting the change we’d like to see. We’re medicos, engineers, builders and designers who’ve felt (on every side of the equation) what non-care feels like, the moral and practical toll as a provider or receiver.

Building what we’re building isn’t always easy. Standard tech playbooks routinely fail in healthcare, and the friction of the medical system will test your perseverance. But we didn’t choose easy, we chose a purpose-driven mission that actually matters. We hold ourselves to a higher standard because healthcare demands it. If you join Heidi, you recognize that the deeper question isn’t whether AI can solve the global healthcare crisis, but whose hands will shape it.

True A-players progress extremely fast here. The nature of the scale-up game is demanding, but we value sustainable performance and mental health. You're trusted to perform, and you set your schedule. We operate on outcomes > inputs, not process theatre. We all take the bins out, metaphorically and literally.

Heidi’s commitment to Diversity, Equity and Inclusion

Heidi is dedicated to creating an equitable, inclusive, and supportive work environment that brings people together from diverse backgrounds, experiences, and perspectives. Our strength is in our differences. We're proud to be an equal opportunity employer and are proud to welcome all applicants as we're committed to promoting a culture of opportunity for all.