Senior AI Engineer

MD Ally; United States; Remote; USD 150,000 - 250,000; Full time

Benefits offered by this job

Salary and equity package

Full medical, dental & vision coverage

401(k) plan and flexible PTO

Job summary

MD Ally seeks a Senior AI Engineer to architect, build, and scale production AI systems for clinical use. You will implement HIPAA‑compliant data pipelines, guardrails, and real‑time speech/text processing integrated into telehealth and EHRs.

You’ll own ambient transcription, EHR read/write agents, and RAG‑based chart summarization, plus patient intake, triage routing, and post‑visit care plan automation with secure, auditable software.

Qualifications

• 5+ years of software engineering experience, with 2+ years shipping production LLM, NLP, or speech‑to‑text applications.
• Strong Engineering Fundamentals: High proficiency in Python and TypeScript/Node.js, with API design, state machines, asynchronous workers, and CI/CD.
• Hands‑On Applied AI Stack: Practical experience with LLM orchestration frameworks, RAG pipelines, vector databases, and structured output validation.
• Security and Privacy Mindset: Experience building HIPAA/SOC2 compliant systems with BAAs and encrypted data flows.
• Cross‑Functional Collaboration: Partner with clinicians, product managers, and security teams to translate medical protocols into deterministic software.

Responsibilities

• Architect and scale production AI systems handling PHI with auditable, private pipelines.
• Build ambient clinical scribes and EHR read/write agents using standards like FHIR/HL7.
• Develop retrieval‑augmented generation systems to summarize complex medical records with source citations.
• Create dynamic symptom intake, triage routing, and post‑visit care plan automation.
• Implement guardrails, audit logs, and real‑time streaming for telehealth workflows.

Skills

Python

TypeScript/Node.js

APIs design

CI/CD

LLM orchestration

RAG pipelines

Vector databases

Security & privacy

Cross‑functional collaboration

Tools

LangGraph

LlamaIndex

Custom agent harnesses

pgvector

Pinecone

Job description

About the Role

MD Ally integrates with public-safety systems so eligible non-emergency 911 callers can reach care from a clinician instead of defaulting to the highest-cost transport and emergency department pathway. That adds a clinical step before a member enters that pathway. We're looking for a Senior AI Engineer to architect, build, and scale our production AI systems.

These systems handle PHI and support clinical decisions, so they have to be accurate, private, and auditable. You'll build the infrastructure that makes that true: deterministic guardrails, HIPAA-compliant data pipelines, clinical evaluation suites, and real-time voice and text workflows integrated into our telehealth platform and Electronic Health Records (EHR).

What You Will Build and Own

Clinician Productivity and EHR Automation

Doctors and nurses spend hours clicking through Electronic Health Records. You'll build tools to automate that administrative work:

• Ambient Clinical Scribes: Build audio-to-text pipelines using models like Whisper or Deepgram combined with LLMs that listen to telehealth video visits, filter out small talk, and draft structured SOAP notes (Subjective, Objective, Assessment, Plan).
• EHR Read/Write Agents: Build integrations using FHIR or HL7 standards that pull a patient's past labs and medications before a visit, and push approved clinical notes, ICD-10 billing codes, or draft prescriptions back into systems like Epic, Athenahealth, or Elation.
• Pre-Visit Chart Summarization: Architect Retrieval-Augmented Generation (RAG) systems that digest dozens of pages of messy PDF medical records or specialist referrals into clean, concise summaries with exact source citations for the provider.

Patient‑Facing Intake and Care Navigation

Before or after a virtual visit, patients interact with chat or voice interfaces. You'll build the backend logic that keeps these interactions helpful and safe:

• Dynamic Symptom Intake: Build conversational agents that gather a patient's medical history and symptoms before a video call, and ask follow‑up questions based on clinical protocols.
• Smart Triage and Routing: Develop classification systems that analyze patient messages to route urgent red‑flag symptoms (such as chest pain) immediately to an on‑call nurse, while routing routine prescription refills or scheduling requests to automated workflows.
• Post‑Visit Follow‑Up and Care Plans: Create automated systems that translate complex medical jargon from a doctor's note into plain‑language instructions, medication reminders, and check‑ins in the patient's preferred language.

Clinical Safety, Privacy, and Guardrail Infrastructure

Because we handle Protected Health Information (PHI) and human health, you'll build the safety infrastructure around every model:

• PHI Redaction and Zero‑Retention Pipelines: Build middleware that strips names, dates of birth, and SSNs before data reaches external APIs, routing all traffic strictly through HIPAA‑compliant endpoints (such as AWS Bedrock, Azure OpenAI, or GCP Vertex AI) under a Business Associate Agreement (BAA).
• Deterministic Clinical Guardrails: Implement real‑time validation layers that block models from diagnosing conditions, recommending unapproved treatments, or hallucinating drug dosages, forcing standard fallback responses whenever clinical boundaries are crossed.
• Real‑Time Audio/Video Processing: Engineer low‑latency streaming infrastructure using WebSockets and WebRTC so voice agents and live transcription tools respond in milliseconds during live telehealth calls without awkward pauses.

Clinical Evaluation and Observability Systems

We don't ship a prompt or model change without proving it's safe. You'll build automated testing harnesses:

• Synthetic Patient Simulators: Build AI agents programmed to act like patients with specific conditions, accents, or edge‑case behaviors to stress‑test intake workflows before real patients interact with them.
• LLM‑as‑a‑Judge and Clinical Grading Pipelines: Build automated test suites that grade generated SOAP notes against gold‑standard physician notes, checking for omitted allergies, incorrect dosages, or hallucinated symptoms.
• Audit Logs and Traceability: Maintain full logging systems so that if a clinician or regulator asks why the system suggested an output, our team can trace the exact prompt, retrieved medical document, and model version used.

What We're Looking For

• 5+ years of software engineering experience, with 2+ years shipping production LLM, NLP, or speech‑to‑text applications.
• Strong Engineering Fundamentals: High proficiency in Python and TypeScript/Node.js, with deep experience designing reliable APIs, state machines, asynchronous workers, and CI/CD pipelines.
• Hands‑On Applied AI Stack: Practical experience with LLM orchestration frameworks (such as LangGraph, LlamaIndex, or custom agent harnesses), RAG pipelines, vector databases (such as pgvector or Pinecone), and structured output validation.
• Security and Privacy Mindset: Experience building systems under strict compliance frameworks (HIPAA, SOC2) and working with Business Associate Agreements (BAAs) and encrypted data flows.
• Cross‑Functional Collaboration: Ability to partner directly with clinicians, Product Managers, and Security teams to translate medical protocols into deterministic, testable software.

Preferences

• Direct experience in digital health, telehealth, or health‑tech startups.
• Familiarity with healthcare data standards (FHIR, HL7), medical coding (ICD‑10, CPT), or EHR integrations (Epic, Athenahealth, Elation).
• Experience with real‑time audio processing and medical transcription models (such as Whisper, Deepgram Medical, or AWS Transcribe Medical).

Benefits:

Salary and equity package, full medical, dental, and vision coverage, 401(k) plan, and flexible PTO.