Staff AI Engineering

25madison LLC

United States

Remote

USD 150,000 - 220,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Competitive salary + equity
Health insurance
Remote-first, async-friendly
Small team shipping to production

Job summary

Basalt Health is hiring Staff AI Engineer to own the intelligence layer of our healthcare platform, coordinating LLM calls, retrieval pipelines, and evaluation frameworks. You will ship across the stack, pushing AI infrastructure with a focus on reliability and safety.

You’ll work on LangGraph orchestration, embeddings, and tracing to prevent hallucinations, while collaborating in a remote-first, async-friendly environment with a small, production-focused team.

Qualifications

  • Hands-on LLM experience with major APIs (OpenAI, Anthropic, or Google) and understanding of context windows.

Responsibilities

  • Own the intelligence layer: orchestration of LLM calls across pipelines.
  • Build retrieval pipelines using embeddings and vector similarity.
  • Set up tracing to debug why a summarization chain hallucinates.
  • Fine-tune models on medical terminology or clinical note structure.
  • Evaluate Gemini vs Claude vs GPT for healthcare tasks.
  • Design data pipelines turning unstructured clinical docs into searchable vectors.
  • Write Python and TypeScript services tying components together.

Skills

LLM orchestration
LangGraph
LangChain
Python
TypeScript
Vector search

Tools

PostgreSQL
BigQuery
FHIR
GCP
Terraform

Job description

Staff AI Engineering

Remote US

Engineering

Remote

Full-time

Who We Are

We're building tools that help healthcare teams work faster and smarter. Our software handles the tedious parts of patient care coordination so that clinicians can focus on patients. We ship fast, iterate constantly, and believe AI should amplify human capability, not replace it.

The Role

You’ll own the intelligence layer of our platform—the orchestration systems that coordinate LLM calls, the retrieval pipelines that surface relevant medical history, and the evaluation frameworks that keep it all honest.

You’re a generalist who can and will ship across the stack, but you geek out on AI infrastructure. You’ve opinions on prompt management, you’ve debugged token limits at 2am, and you know that "it works in the playground" means nothing. You’re pushing the limits of your bot swarm farther every day to maximize your impact and excited for the future.

You Might:
  • Improve our LangGraph orchestration to handle complex clinical workflows
  • Build retrieval pipelines that search patient records using embeddings and vector similarity
  • Set up Langfuse/LangSmith tracing to debug why a summarization chain is hallucinating
  • Fine‑tune a model on medical terminology or clinical note structure
  • Evaluate Gemini vs Claude vs GPT for specific healthcare tasks
  • Design the data pipeline that turns unstructured clinical docs into searchable vectors reliably and responsibly
  • Write the TypeScript or Python service(s) that ties it all together
You Should Have
  • Hands‑on LLM experience. You've built with OpenAI, Anthropic, or Google APIs. You understand context windows, temperature, and when to use which model.
  • Orchestration chops. LangChain, LangGraph, or similar. You know how to chain calls, handle failures, and manage state.
  • Observability instincts. You've used Langfuse, LangSmith, or Phoenix. You know that production AI without tracing is flying blind.
  • Strong hold of agentic code authoring systems. You are pushing the limits of Claude, Codex, Gemini, Copilot, or similar daily.
  • Retrieval/RAG experience. You've built vector search with Pinecone, Weaviate, pgvector, or Vertex AI Matching Engine. You understand chunking strategies, embedding models, and reranking.
  • Generalist coding ability. You can write Python for ML pipelines and TypeScript for services. You're not afraid of infrastructure and you're not too caught up in the moment to pitch in on the boring stuff.
Bonus Points
  • Fine‑tuning experience (LoRA, PEFT, or full fine‑tunes)
  • Healthcare/FHIR/clinical data background
  • Experience with Gemini models and Vertex AI
  • Evaluation frameworks (RAGAS, custom evals, human‑in‑the‑loop)
  • You've made embeddings work on messy, real‑world data
Tech Stack
  • AI/ML: LangGraph, Vertex AI, Gemini, embeddings, vector search
  • Observability: Langfuse, LangSmith
  • Backend: Python (FastAPI), TypeScript (Fastify)
  • Data: PostgreSQL, BigQuery, FHIR
  • Infrastructure: GCP (Cloud Run, Pub/Sub), Terraform
What We Offer
  • Competitive salary + equity
  • Health insurance (we're a healthcare company—we get it)
  • Remote‑first, async‑friendly
  • Small team where your work ships to production, not a backlog

Basalt Health is an equal opportunity employer.

We are only able to consider candidates who are already authorized to work in the United States without employer sponsorship.

Compensation

The base pay range for this role is $150,000 – $220,000 per year.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff AI Engineer - Remote, Equity & Health Benefits
Staff AI Engineer - Remote, Equity & Health Benefits

25madison LLC • United States

Remote
USD 150,000 - 220,000
Competitive salary + equity
Health insurance
Remote-first, async-friendly
+1
Senior Software Engineer
Senior Software Engineer

Marvin Behavioral Health • Colorado

Hybrid
USD 160,000 - 190,000
Equity
Health, dental, and vision
401(k)
+3
Senior AI Engineer
Senior AI Engineer

MD Ally • United States

Remote
USD 150,000 - 250,000
Salary and equity package
Full medical, dental & vision coverage
401(k) plan and flexible PTO
Senior Fullstack Engineer, AI Observability & Evals Platform
Senior Fullstack Engineer, AI Observability & Evals Platform

Langchain • Cambridge (MA)

On-site
USD 175,000 - 240,000
Medical coverage
Dental coverage
Vision coverage
+3
Senior Fullstack Engineer, AI Observability & Evals Platform
Senior Fullstack Engineer, AI Observability & Evals Platform

FDE Times • Boston (MA)

On-site
USD 175,000 - 240,000
Medical, dental, and vision coverage
Flexible vacation
401(k) plan
+2
Production AI Agent Engineer – Early Career
Production AI Agent Engineer – Early Career

LangChain, Inc. • San Francisco (CA)

On-site
USD 155,000 - 165,000
Medical benefits
401(k) plan
Meals on in-office days
Senior Frontend Engineer for AI Observability Platform
Senior Frontend Engineer for AI Observability Platform

LangChain, Inc. • Boston (MA)

Hybrid
USD 175,000 - 240,000
Medical, dental, and vision coverage
401(k) plan
Flexible vacation
Senior Backend Software Engineer, AI Observability & Evals Platform (LangSmith)
Senior Backend Software Engineer, AI Observability & Evals Platform (LangSmith)

Langchain • Cambridge (MA)

On-site
USD 180,000 - 240,000
Medical insurance
Dental insurance
Vision insurance
+3
AI Engineer
AI Engineer

harriscomputer • United States

Remote
USD 100,000 - 150,000
Remote work
Office optional
Senior Software Engineer
Senior Software Engineer

Marvin • United States

Hybrid
USD 160,000 - 190,000
Health, dental, and vision
Equity
401(k)
+2