Senior AI Engineer — Clinical AI & Production Systems

Aimlroles; Northern (KY); Hybrid; USD 190,000 - 250,000; Full time

Benefits offered by this job

Health benefits fully covered

OneMedical membership for employees &/

Fertility & family planning

401k with match

Flexible Time Off

CME/CEU support

Job summary

Atria Health Institute seeks a Senior AI Engineer to own the design, development, and production deployment of LLM-powered systems that support clinicians and members: agentic medical companions, clinical decision support, and document understanding pipelines.

You will lead end-to-end AI production from orchestration to evaluation, focusing on safety, observability, latency, and HIPAA compliance, partnering with clinicians and data teams to translate biomedical requirements into reliable,

Qualifications

• Hands-on experience building and operating LLM-powered or production ML systems (4+ years).
• Strong Python skills and production backend framework experience (FastAPI, Flask).
• Experience with LLM orchestration, function/tool calling, and retrieval system design.
• Cloud data warehouse knowledge (Snowflake preferred) and modern data tooling (dbt, Dagster, Airflow).
• Track record designing evaluation systems for LLM apps with regression testing and safe deployments.
• Familiarity with safety considerations for LLM apps: guardrails and adversarial testing.
• Understanding HIPAA, BAAs, PHI handling in regulated environments.
• Strong written communication and ability to document complex systems.

Responsibilities

• Architect and ship production AI systems end-to-end (orchestration, retrieval, inference, evaluation, monitoring) for clinical decision support, document understanding, and agentic medical companions.
• Design agentic workflows with frontier LLMs and self-hosted models, avoiding unnecessary agentization.
• Lead development of structured information extraction pipelines from clinical documents with verification gates and human-in-the-loop checks.
• Build robust evaluation infrastructure (golden datasets, LLM-as-judge, regression tests, online A/B testing) with emphasis on accuracy, safety, cost, and latency.
• Engineer for latency and cost: streaming, caching, prompt compression, model routing, and budgets.
• Own production health: observability, prompt versioning, drift detection, cost monitoring, and PHI-safe logging.
• Operationalize models from the AI Scientist team: serving, evaluation in production, rollback paths, and feedback loops for training data.
• Partner with clinicians, data engineers, and researchers to translate clinical requirements into deliverable systems.
• Educate and mentor PMs/engineers on applied AI best practices and model selection.

Skills

LLM orchestration

Python

Production backend (FastAPI/Flask)

Retrieval systems

Model evaluation & observability

PHI safety & HIPAA

Technical communication

AI vendor diligence

Tools

Snowflake

dbt

Dagster

Airflow

vLLM/Triton

Job description

Atria Health Institute seeks a Senior AI Engineer to own the design, development, and production deployment of LLM-powered systems that support clinicians and members: agentic medical companions, clinical decision support, and document understanding pipelines.

You will lead end-to-end AI production from orchestration to evaluation, focusing on safety, observability, latency, and HIPAA compliance, partnering with clinicians and data teams to translate biomedical requirements into reliable,