Senior AI Engineer

MD Ally

United States

Remote

USD 180,000 - 240,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Salary and equity package
Full medical, dental, and vision
401(k) plan
Flexible PTO

Job summary

MD Ally is seeking a Senior AI Engineer to architect, build, and scale production AI systems for telehealth. You will own end-to-end features, from rapid prototyping to deployment, with a focus on HIPAA-compliant data pipelines and real-time workflows integrated with EHR systems.

You will work on ambient clinical scribe, EHR read/write agents, and RAG-based patient care summaries, ensuring accuracy, privacy, and auditable processes across hospital-grade platforms.

Qualifications

  • 5+ years of software engineering experience.
  • 2+ years shipping production LLM, NLP, or speech-to-text apps.
  • Strong fundamentals in Python and TypeScript/Node.js.
  • Experience building reliable APIs, state machines, and CI/CD pipelines.
  • Hands-on with LLM orchestration frameworks and RAG pipelines.

Responsibilities

  • Architect, build, and scale production AI systems with guardrails.
  • Design retrieval pipelines and evaluation infrastructure.
  • Own full lifecycle from prototyping to deployment, observability, and cost optimization.
  • Implement PHI Redaction and HIPAA-compliant data handling pipelines.
  • Collaborate with clinicians, Product Managers, and Security teams.

Skills

Python
TypeScript
API design
State machines
CI/CD
LLM orchestration
NLP

Tools

LangGraph
LlamaIndex
Pinecone
pgvector

Job description

MD Ally Senior AI Engineer Remote · Full time

We are seeking an AI Engineer to architect, build, and scale our production AI systems. You will move beyond simple API wrappers to design reliable agentic workflows, retrieval pipelines, and evaluation infrastructure. You will own the full lifecycle of AI features, including rapid prototyping to deployment, observability, and cost optimization.


About MD Ally

MD Ally is a 9-1-1 Telehealth solution that enables greater connectivity between public health & safety and the broader healthcare ecosystem. Our solutions allow dispatchers and first responders to transfer non-emergency callers from 911 to virtual care instead of dispatching valuable, emergency resources.


Description

About the Role


MD Ally integrates with public-safety systems so eligible non-emergency 911 callers can reach care from a clinician instead of defaulting to the highest-cost transport and emergency department pathway. That adds a clinical step before a member enters that pathway. We're looking for a Senior AI Engineer to architect, build, and scale our production AI systems.


These systems handle PHI and support clinical decisions, so they have to be accurate, private, and auditable. You'll build the infrastructure that makes that true: deterministic guardrails, HIPAA-compliant data pipelines, clinical evaluation suites, and real-time voice and text workflows integrated into our telehealth platform and Electronic Health Records (EHR).


What You Will Build and Own


Clinician Productivity and EHR Automation


Doctors and nurses spend hours clicking through Electronic Health Records. You'll build tools to automate that administrative work:



  • Ambient Clinical Scribes: Build audio-to-text pipelines using models like Whisper or Deepgram combined with LLMs that listen to telehealth video visits, filter out small talk, and draft structured SOAP notes (Subjective, Objective, Assessment, Plan).

  • EHR Read/Write Agents: Build integrations using FHIR or HL7 standards that pull a patient's past labs and medications before a visit, and push approved clinical notes, ICD-10 billing codes, or draft prescriptions back into systems like Epic, Athenahealth, or Elation.

  • Pre-Visit Chart Summarization: Architect Retrieval-Augmented Generation (RAG) systems that digest dozens of pages of messy PDF medical records or specialist referrals into clean, concise summaries with exact source citations for the provider.


Patient-Facing Intake and Care Navigation


Before or after a virtual visit, patients interact with chat or voice interfaces. You'll build the backend logic that keeps these interactions helpful and safe:



  • Dynamic Symptom Intake: Build conversational agents that gather a patient's medical history and symptoms before a video call, and ask follow-up questions based on clinical protocols.

  • Smart Triage and Routing: Develop classification systems that analyze patient messages to route urgent red-flag symptoms (such as chest pain) immediately to an on-call nurse, while routing routine prescription refills or scheduling requests to automated workflows.

  • Post-Visit Follow-Up and Care Plans: Create automated systems that translate complex medical jargon from a doctor's note into plain-language instructions, medication reminders, and check-ins in the patient's preferred language.


Clinical Safety, Privacy, and Guardrail Infrastructure


Because we handle Protected Health Information (PHI) and human health, you'll build the safety infrastructure around every model:



  • PHI Redaction and Zero-Retention Pipelines: Build middleware that strips names, dates of birth, and SSNs before data reaches external APIs, routing all traffic strictly through HIPAA-compliant endpoints (such as AWS Bedrock, Azure OpenAI, or GCP Vertex AI) under a Business Associate Agreement (BAA).

  • Deterministic Clinical Guardrails: Implement real-time validation layers that block models from diagnosing conditions, recommending unapproved treatments, or hallucinating drug dosages, forcing standard fallback responses whenever clinical boundaries are crossed.

  • Real-Time Audio/Video Processing: Engineer low-latency streaming infrastructure using WebSockets and WebRTC so voice agents and live transcription tools respond in milliseconds during live telehealth calls without awkward pauses.


Clinical Evaluation and Observability Systems


We don't ship a prompt or model change without proving it's safe. You'll build automated testing harnesses:



  • Synthetic Patient Simulators: Build AI agents programmed to act like patients with specific conditions, accents, or edge-case behaviors to stress-test intake workflows before real patients interact with them.

  • LLM-as-a-Judge and Clinical Grading Pipelines: Build automated test suites that grade generated SOAP notes against gold-standard physician notes, checking for omitted allergies, incorrect dosages, or hallucinated symptoms.

  • Audit Logs and Traceability: Maintain full logging systems so that if a clinician or regulator asks why the system suggested an output, our team can trace the exact prompt, retrieved medical document, and model version used.


What We're Looking For



  • 5+ years of software engineering experience, with 2+ years shipping production LLM, NLP, or speech-to-text applications.

  • Strong Engineering Fundamentals: High proficiency in Python and TypeScript/Node.js, with deep experience designing reliable APIs, state machines, asynchronous workers, and CI/CD pipelines.

  • Hands-On Applied AI Stack: Practical experience with LLM orchestration frameworks (such as LangGraph, LlamaIndex, or custom agent harnesses), RAG pipelines, vector databases (such as pgvector or Pinecone), and structured output validation.

  • Security and Privacy Mindset: Experience building systems under strict compliance frameworks (HIPAA, SOC2) and working with Business Associate Agreements (BAAs) and encrypted data flows.

  • Cross-Functional Collaboration: Ability to partner directly with clinicians, Product Managers, and Security teams to translate medical protocols into deterministic, testable software.


Preferences



  • Direct experience in digital health, telehealth, or health-tech startups.

  • Familiarity with healthcare data standards (FHIR, HL7), medical coding (ICD-10, CPT), or EHR integrations (Epic, Athenahealth, Elation).

  • Experience with real-time audio processing and medical transcription models (such as Whisper, Deepgram Medical, or AWS Transcribe Medical).


Benefits: Salary and equity package, full medical, dental, and vision coverage, 401(k) plan, and flexible PTO.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Engineer
Senior AI Engineer

MD Ally • United States

Remote
USD 150,000 - 250,000
Salary and equity package
Full medical, dental & vision coverage
401(k) plan and flexible PTO
Senior Applied AI Engineer, Patient Experience
Senior Applied AI Engineer, Patient Experience

Bask Health • New York (NY)

On-site
USD 180,000 - 240,000
Senior AI Engineer
Senior AI Engineer

Ellipsis Health • San Francisco (CA)

Hybrid
USD 160,000 - 210,000
401(k) matching
Health insurance
Vision insurance
+2
Senior Applied AI Engineer, Patient Experience
Senior Applied AI Engineer, Patient Experience

Bask Health • United States

On-site
USD 140,000 - 210,000
Full Stack AI Engineer
Full Stack AI Engineer

CareTria, Inc. • United States

On-site
USD 140,000 - 210,000
Applied ML Engineer, Care Experience
Applied ML Engineer, Care Experience

Bask Health • New York (NY)

On-site
USD 120,000 - 170,000
Applied AI Engineer
Applied AI Engineer

Norbert Health • New York (NY)

On-site
USD 100,000 - 150,000
Equity participation
Competitive salary
High autonomy and technical ownership
Applied ML Engineer, Care Experience
Applied ML Engineer, Care Experience

Bask Health • United States

Remote
USD 120,000 - 180,000
Applied AI Engineer, Patient Experience
Applied AI Engineer, Patient Experience

Bask Health • United States

Remote
USD 120,000 - 180,000
AI Engineering Lead
AI Engineering Lead

Beni • United States

Remote
USD 200,000 - 290,000
Early-stage equity