Staff/Principal AI/ML Engineer

Zus Health; Boston (MA); Hybrid; USD 210,000 - 280,000; Full time

Benefits offered by this job

Health insurance

401k with match

Unlimited PTO

Competitive compensation

Job summary

Zus Health is seeking a Staff/Principal AI/ML Engineer to define how LLMs and AI agents power our platform. You will own architecture, tooling, and product roadmap, mentoring engineers and delivering production-grade capabilities.

You will build scalable retrieval, RAG, and agent systems over clinical data, ensure safety/compliance, and collaborate with product, sales, and clinical teams to drive impact at scale.

Qualifications

• 7+ years building and shipping AI/ML systems.
• Hands-on depth in modern LLM engineering, including tool use and structured outputs.
• Experience with safety, privacy and reliability when handling clinical data.
• Excellent written and verbal communication with clinicians and product leaders.

Responsibilities

• Architect the LLM platform and shared infra for AI features.
• Build agentic systems that reason over patient data with tool-using agents.
• Own retrieval over clinical data with RAG, embeddings and grounding.
• Establish evaluation pipelines, offline/online evals and CI for models and prompts.
• Drive production CI/CD, observability, and human-in-the-loop feedback loops.
• Prototype with providers, product managers and engineers to ship fast and iterate.
• Set and uphold engineering standards and documentation.

Skills

Python

PyTorch

LLM engineering

Hugging Face

MLOps

Education

BS in Computer Science, Statistics or related field

Tools

AWS

Docker

Kubernetes

CI/CD pipelines

Job description

Zus Health is an AI-native healthcare data platform that brings fragmented patient information together to make it complete, current, and actionable. Through our nationwide data network, platform, and intelligent workflows, we help providers, value-based care organizations, payers, life sciences, and digital health companies drive business results, strengthen provider trust, and improve operational efficiency. Founded in 2021 by Jonathan Bush, co-founder and former CEO of athenahealth, Zus is building the foundation for a more connected, precise, and effective healthcare system.

What we’re looking for…

Zus is hiring a Staff/Principal AI/ML Engineer to define how LLMs and AI agents power our products and the AI platform underneath them. You'll be one of the most seasoned AI/ML voices in the building, with real ownership of architecture, tooling and roadmap.
Zus aggregates clinical data from across the healthcare ecosystem. That data is messy, multi-modal and enormously valuable. Modern foundation models finally make it possible to turn it into structured, trustworthy insight at the point of care and across populations. Your job is to make that happen, safely and at scale.

This is a hands-on role. You'll write production code, ship prototypes to real users, set technical direction, and mentor the engineers who build alongside you. This work will require interactions with solution engineers, product managers, sales, and marketing.

As part of our team, you will...

Architect our LLM platform. Design the shared infrastructure every AI feature at Zus runs on: model gateway and routing across frontier and open-weight models, prompt and version management, caching, cost and latency controls, and PHI-safe guardrails.

Build agentic systems. Ship tool-using agents that reason over a patient's longitudinal record by summarizing histories, reconciling medications, surfacing care gaps and answering clinician questions with cited evidence. Use patterns like MCP servers, function calling and multi-step planning.

Own retrieval over clinical data. Build RAG and hybrid search over FHIR resources and unstructured notes: chunking and embedding strategies for clinical text, vector and graph retrieval, reranking, and grounding so every answer traces back to its source document.

Make evaluation a first-class system. Stand up offline and online evals: golden datasets with clinical reviewers, LLM-as-judge pipelines, regression suites in CI, and production monitoring for hallucination, drift and safety.

Adapt models when prompting isn't enough. Fine-tune and distill (LoRA/PEFT, SFT, preference optimization) for clinical extraction, coding and normalization tasks where smaller, cheaper, specialized models win.

Run AI/ML in production. Build CI/CD for models and prompts, inference serving, observability and human-in-the-loop feedback loops that turn user corrections into better models.

Prototype with users. Work directly with providers, product managers and engineers to find high-leverage problems, ship prototypes fast, and iterate on real usage.

Set the bar. Lead design reviews, write the docs and standards others build on, mentor engineers, and keep Zus current as the AI landscape moves.

What you bring…

7+ years building and shipping AI/ML systems, including 2+ years putting LLM-powered products into production for real users.

Hands-on depth in modern LLM engineering: RAG, agents and tool use, structured outputs, prompt and context engineering, and the trade-offs between frontier APIs and open-weight models.

A track record of building ML infrastructure others depend on — serving, pipelines, model and prompt versioning, CI/CD, monitoring.

A rigorous approach to evaluation: you've designed eval sets and metrics, and you don't ship on vibes.

Strong Python and software engineering fundamentals; fluency in PyTorch and the Hugging Face ecosystem.

Production experience on a major cloud (AWS preferred) with containers, and infrastructure as code.

Sound judgment about safety, privacy and reliability when models touch sensitive data.

Clear writing and communication; you can explain model behavior and risk to clinicians, product leaders and executives.

BS in Computer Science, Statistics or a related field, or equivalent experience.

Are in the Boston metro and excited to collaborate in person on a hybrid schedule, or are remote (EST/CST) but willing to travel for in person collaboration occasionally.

Healthcare or clinical NLP experience, especially with FHIR, C-CDA, clinical terminologies or HIPAA-regulated data.

Fine-tuning, distillation or preference optimization of LLMs at meaningful scale.

Built MCP servers or multi-agent systems used in production.

Designed and executed pre-training of transformer models.

Open-source contributions, publications or talks in ML, NLP or LLM systems.

Experience as the first or founding ML engineer at a growth-stage company.

We will offer you…

• Competitive compensation that reflects the value you bring to the team a combination of cash and equity
• Robust benefits that include health insurance, wellness benefits, 401k with a match, unlimited PTO
• Opportunity to work alongside a passionate team that is determined to help change the world (and have fun doing it)

Please Note: Research shows that candidates from underrepresented backgrounds often don’t apply unless they meet 100% of the job criteria. While we have worked to consolidate the minimum qualifications for each role, we aren’t looking for someone who checks each box on a page; we’re looking for active learners and people who care about disrupting the current healthcare system with their unique experiences.