Machine Learning Engineer

Modernizing Medicine, Inc.; United States; Hybrid; USD 100,000 - 140,000; Full time

Benefits offered by this job

Comprehensive medical, dental, and vision benefits

401(k) matching contribution

Generous Paid Time Off

Company paid Life and Disability benefits

Flexible Spending Account

Professional development opportunities

Dog-friendly HQ

Job summary

Modernizing Medicine, Inc. seeks a Machine Learning Engineer to enhance its patient engagement platform with autonomous AI agents. This role involves developing systems that perform complex tasks autonomously while ensuring safety and empathy.

The ideal candidate has over 3 years of machine learning experience, strong Python skills, and hands-on knowledge of AWS and AI frameworks like LangGraph. The position offers comprehensive benefits, including flexible working arrangements and professional development opportunities.

Qualifications

• 3+ years of experience in Machine Learning, focusing on AI agents and conversational systems.
• Proven experience in building autonomous systems using no-code/low-code frameworks.
• Hands-on experience with fine-tuning models and evaluating frameworks.

Responsibilities

• Design and deploy multi-agent systems for clinical workflows.
• Build and optimize conversational AI interfaces for patient engagement.
• Implement evaluation frameworks to measure agent performance.

Skills

Machine Learning

Conversational NLP

Python

AWS

Databricks

CI/CD Pipelines

Tools

LangGraph

AutoGen

CrewAI

RAGAS

DeepEval

Job description

Job Description Summary

We are seeking a Machine Learning Engineer to build the “active brain” of our patient engagement platform. In this role, you will develop autonomous and conversational AI agents capable of triaging patient messages and managing clinical workflows. Your mission is to design systems that don’t just respond, but act—executing complex tasks with minimal human intervention while maintaining high standards of safety, empathy, and accuracy.

What you’ll do

• Autonomous Agent Development – Design and deploy multi-agent systems using frameworks such as LangGraph, AutoGen, CrewAI, or no-code/low-code agent frameworks to execute complex, non-linear clinical workflows.
• Patient Engagement Agents – Build and optimize conversational AI interfaces that handle message triage, sentiment analysis, and context‑aware responses that are natural, empathetic, and patient‑centric. Develop tool‑calling architectures that enable AI agents to interact with external APIs, EHR systems, and databases to perform real‑world actions.
• Model & Agent Evaluation – Design and implement evaluation frameworks to measure model and agent performance, including accuracy, reliability, safety, and task completion. Continuously improve systems using structured evaluation methodologies and feedback loops.
• Safety & Guardrails – Implement robust evaluation pipelines and guardrails (e.g., NeMo Guardrails) to ensure agent safety, clinical accuracy, and HIPAA compliance.
• AI Observability – Create and maintain dashboards to monitor agent reasoning traces, system performance, uptime, and task completion rates in production environments.

What you’ll bring

• 3+ years of experience in Machine Learning with a strong focus on AI agents and conversational NLP systems.
• Proven experience building autonomous systems using LangGraph, AutoGen, CrewAI, or similar no-code/low-code agent frameworks.
• Hands‑on experience with fine‑tuning models and implementing rigorous evaluation frameworks to ensure agent performance, safety, and reliability.
• Familiarity with techniques for detecting hallucinations, bias, and failure modes in agent behavior.
• Strong understanding of guardrails, alignment techniques, and safety mechanisms to ensure compliant, accurate, and trustworthy agent outputs in sensitive environments.
• Hands‑on experience with AWS and Databricks, including building CI/CD pipelines for deploying machine learning models and AI agents.
• Strong proficiency in Python and experience with modern evaluation frameworks such as RAGAS or DeepEval.

ModMed Benefits Highlight

• United States Comprehensive medical, dental, and vision benefits, including a company Health Savings Account contribution.
• 401(k): ModMed provides a matching contribution each payday of 50% of your contribution deferred on up to 6% of your compensation. After one year of employment, 100% of any matching contribution you receive is yours to keep.
• Generous Paid Time Off and Paid Parental Leave programs.
• Company paid Life and Disability benefits.
• Flexible Spending Account and Employee Assistance Programs.
• Company‑sponsored Business Resource & Special Interest Groups that provide engaged and supportive communities within ModMed.
• Professional development opportunities, including tuition reimbursement programs and unlimited access to LinkedIn Learning.
• Global presence and in‑person collaboration opportunities; dog‑friendly HQ (US).
• Hybrid office‑based roles and remote availability for some roles.
• Weekly catered breakfast and lunch, treadmill workstations, Zen, and wellness rooms within our headquarters.