Voice AI Agent Engineer

OhMD; United States; Hybrid; USD 110,000 - 170,000; Full time

Job summary

OhMD is seeking an engineer to build and optimize a high-quality voice and chat agent platform for pediatric practices. You will configure end-to-end agent behavior, design conversation flows, and integrate with major EHR systems to keep data flowing smoothly.

Expect three-week go-lives, rigorous testing, and a focus on patient safety. You will work across engineering, implementation, and customer success to deliver reliable AI-enabled patient interactions.

Qualifications

• Built and deployed at least one LLM-powered conversational agent with prompts, flows and tool definitions.

Responsibilities

• Build agent behavior: design conversation flows, prompts, tool definitions and escalation logic.
• Own go-lives: scope agent tasks, handle non-standard paths, ensure smooth first-week live run.
• Measure quality: listen to real calls, create evals and regression tests, fix root causes.
• Design for messy data: handle data from various EHRs, ensure resilience when lookups fail.
• Turn projects into templates: document reusable workflows and hand to implementations.

Skills

LLM convo agent

Voice agent design

Tool schemas

REST APIs

Agent testing

Latency management

Cross-functional

HIPAA data

EHR integration

Tools

Vapi

Bland

LiveKit Agents

Pipecat

curl

Postman

Job description

Location: Burlington, VT (hybrid) or remote

ABOUT OHMD

OhMD is patient communication software used by thousands of physician practices: two-way texting, digital forms, automated workflows, and OhMD AI, our voice and text agent. OhMD AI answers on the first ring, handles the routine calls a front desk shouldn't have to take, and hands the conversation to a human the moment it needs one, with full context attached. We integrate with 85+ EHRs, and practices running OhMD see 68% fewer calls reach their staff.

WHY THIS ROLE EXISTS

We don't believe you can automate a truly human interaction. Today's best AI can confidently resolve maybe 60% of what a practice hears in a day. The other 40% is an anxious parent, a complicated refill history, or a patient who needs to be heard rather than routed. Our product is built on knowing the difference.

Every practice is different. A pediatric group handles refills nothing like a dermatology practice, their EHR is configured differently, and their front office has its own unwritten rules about what gets escalated, to whom, and how fast. The agent has to learn all of it and be right, live, on a real call.

You'll build that. This is a hands‑on engineering role: you'll design, configure, test, and debug the agents our practices go live with, and you'll stay on them until calls resolve cleanly. You won't write a spec and hand it to someone else.

WHAT YOU'LL DO

Build agent behavior

You'll turn what you learn from practice staff into working agents on our voice AI platform: conversation flows and state design, prompts, tool definitions, escalation and transfer logic, and fallback paths for when things go wrong. You'll own the configuration end to end.

Own go-lives

Practices go live in about three weeks. You'll scope what the agent handles and what it doesn't, make the calls when a practice doesn't fit the standard path, and make sure its first week live is a good one.

Measure quality honestly

You'll listen to real calls, a lot of them. You'll build evals and regression tests that catch failures before a practice does, read the transcripts where the agent got it wrong, and fix the root cause. When a failure is in the platform or the integration rather than the agent, you'll diagnose it precisely enough that engineering can fix it quickly.

Design for messy data

Scheduling, patient context, and refill data come from athenahealth, eClinicalWorks, Epic, AdvancedMD, ModMed, and a long tail of other EHRs, and that data is rarely tidy. You'll work with our integrations engineers on what the agent needs from each system, and design behavior that holds up when a provider list is stale or a lookup fails.

Turn projects into templates

The first time we solve a workflow it's a project; by the fifth time it should be a reusable setup. You'll notice when something is ready to graduate, document it, hand it to implementation, and move on.

YOUR FIRST 90 DAYS

• Weeks 1–4: Listen to calls, shadow go-lives, learn how a practice actually runs, and find where the agent frustrates people today.
• Weeks 5–8: Own your first agent configurations end to end with support and ship changes to live behavior.
• Weeks 9–12: Own go-lives independently and bring us a point of view on what's breaking at scale and what we should build next.

REQUIRED

• - You have built and deployed at least one LLM-powered conversational agent (voice, chat, or SMS) that real users interacted with, and you personally wrote the prompts, flow logic, and tool definitions. Side projects and freelance or agency work count.
• - You have built voice agents on an LLM voice platform such as Vapi or Bland, or with a framework such as LiveKit Agents or Pipecat.
• - You understand LLM tool use and function calling well: defining tool schemas, handling tool errors and timeouts, and deciding when the model should call a tool versus ask the caller. Familiarity with MCP is a plus.
• - You're comfortable with REST APIs and webhooks. You can read API docs, make calls with curl or Postman, and inspect a JSON payload to work out what went wrong.
• - You have designed systematic tests for agent behavior, such as scripted or simulated callers, regression suites, or LLM-graded evals, rather than relying only on manual spot checks.
• - You understand what makes voice different from chat: latency, speech recognition errors, interruptions and turn-taking, transfers, and callers who don't say what they mean.
• - You make decisions with incomplete information and correct course quickly, and you coordinate across engineering, implementation, and customer success without managing any of them.
• - You care about the patient on the other end of the call, who is often stressed, sick, or worried about someone they love.

PREFERRED

• - Call flow design in a contact center or IVR platform such as Twilio Studio, Five9, or Genesys
• - Experience with healthcare systems or workflows, especially EHR scheduling, HL7, or FHIR, or with another industry where the underlying systems are old and unforgiving
• - Work in or close to a medical practice front office
• - Experience handling PHI or working under HIPAA
• - SQL for querying call and outcome data
• - Twilio or telephony basics such as SIP, DTMF, and call routing