Voice AI Agent Engineer

OhMD; Burlington (VT); On-site; USD 112,000 - 160,000; Full time

Job summary

OhMD is seeking a Voice AI Agent Engineer to design, configure, test, and debug AI agents for patient communications. You will own agent behavior, escalation logic, and go-live readiness, ensuring smooth live experiences across varied EHR integrations.

In this hands-on role, you will measure quality through real-call evaluations, build regression tests, and push for robust performance across voice and channel flows.

Qualifications

• Built and deployed at least one LLM-powered conversational agent with prompts and flows.
• Built voice agents on an LLM voice platform such as Vapi or Bland, or LiveKit/Pipecat.
• Understand tool use, function calling, and when to call tools.
• Comfortable with REST APIs and webhooks; read docs and inspect JSON payloads.
• Write scripts in Python or JavaScript/TypeScript for data pulls and analysis.
• Designed systematic tests for agent behavior, including scripted or simulated calls.
• Understand voice vs. chat: latency, ASR errors, interruptions, transfers.
• Make decisions with incomplete information and coordinate across teams.
• Care about patients on the other end of the call.

Responsibilities

• Build agent behavior including prompts, flows, tool definitions, escalation and fallback logic.
• Own go-lives; scope agent handling and ensure first-week success.
• Measure quality with real-call evaluations, transcripts, and regression tests.
• Design for messy data from multiple EHRs and ensure robust behavior.
• Turn projects into reusable templates and hand to implementation.

Skills

LLM prompts

Tool schemas

REST APIs

Python/JavaScript

Test automation

LLM call handling

Cross-functional coordination

Tools

Vapi

Bland

LiveKit Agents

Pipecat

Twilio Studio

Job description

Job Application for Voice AI Agent Engineer at OhMD Back to jobs New

Voice AI Agent Engineer

Reports to: Chief Technology Officer

Location: Burlington, VT (hybrid) or remote

About OhMD

OhMD is patient communication software used by thousands of physician practices: two-way texting, digital forms, automated workflows, and OhMD AI, our voice and text agent. OhMD AI answers on the first ring, handles the routine calls a front desk shouldn't have to take, and hands the conversation to a human the moment it needs one, with full context attached. We integrate with 85+ EHRs, and practices running OhMD see 68% fewer calls reach their staff.

Why this role exists

We don't believe you can automate a truly human interaction. Today's best AI can confidently resolve maybe 60% of what a practice hears in a day. The other 40% is an anxious parent, a complicated refill history, or a patient who needs to be heard rather than routed. Our product is built on knowing the difference.

Every practice is different. A pediatric group handles refills nothing like a dermatology practice, their EHR is configured differently, and their front office has its own unwritten rules about what gets escalated, to whom, and how fast. The agent has to learn all of it and be right, live, on a real call.

You'll build that. This is a hands-on engineering role: you'll design, configure, test, and debug the agents our practices go live with, and you'll stay on them until calls resolve cleanly. You won't write a spec and hand it to someone else.

What you'll do

Build agent behavior. You'll turn what you learn from practice staff into working agents on our voice AI platform: conversation flows and state design, prompts, tool definitions, escalation and transfer logic, and fallback paths for when things go wrong. You'll own the configuration end to end.

Own go-lives. Practices go live in about three weeks. You'll scope what the agent handles and what it doesn't, make the calls when a practice doesn't fit the standard path, and make sure its first week live is a good one.

Measure quality honestly. You'll listen to real calls, a lot of them. You'll build evals and regression tests that catch failures before a practice does, read the transcripts where the agent got it wrong, and fix the root cause. When a failure is in the platform or the integration rather than the agent, you'll diagnose it precisely enough that engineering can fix it quickly.

Design for messy data. Scheduling, patient context, and refill data come from athenahealth, eClinicalWorks, Epic, AdvancedMD, ModMed, and a long tail of other EHRs, and that data is rarely tidy. You'll work with our integrations engineers on what the agent needs from each system, and design behavior that holds up when a provider list is stale or a lookup fails.

Turn projects into templates. The first time we solve a workflow it's a project; by the fifth time it should be a reusable setup. You'll notice when something is ready to graduate, document it, hand it to implementation, and move on.

Your first 90 days

In weeks 1–4, you'll listen to calls, shadow go-lives, learn how a practice actually runs, and find where the agent frustrates people today. In weeks 5–8, you'll own your first agent configurations end to end with support and ship changes to live behavior. In weeks 9–12, you'll own go-lives independently and bring us a point of view on what's breaking at scale and what we should build next.

Required

• You have built and deployed at least one LLM-powered conversational agent (voice, chat, or SMS) that real users interacted with, and you personally wrote the prompts, flow logic, and tool definitions. Side projects and freelance or agency work count.
• You have built voice agents on an LLM voice platform such as Vapi or Bland, or with a framework such as LiveKit Agents or Pipecat.
• You understand LLM tool use and function calling well: defining tool schemas, handling tool errors and timeouts, and deciding when the model should call a tool versus ask the caller. Familiarity with MCP is a plus.
• You're comfortable with REST APIs and webhooks. You can read API docs, make calls with curl or Postman, and inspect a JSON payload to work out what went wrong.
• You can write scripts in Python or JavaScript/TypeScript to pull call data, run test batches, or analyze transcripts. You don't need to ship production application code.
• You have designed systematic tests for agent behavior, such as scripted or simulated callers, regression suites, or LLM-graded evals, rather than relying only on manual spot checks.
• You understand what makes voice different from chat: latency, speech recognition errors, interruptions and turn-taking, transfers, and callers who don't say what they mean.
• You make decisions with incomplete information and correct course quickly, and you coordinate across engineering, implementation, and customer success without managing any of them.
• You care about the patient on the other end of the call, who is often stressed, sick, or worried about someone they love.

Preferred

• You have designed call flows in a contact center or IVR platform such as Twilio Studio, Five9, or Genesys.
• Experience with healthcare systems or workflows, especially EHR scheduling, HL7, or FHIR, or with another industry where the underlying systems are old and unforgiving
• Work in or close to a medical practice front office
• Experience handling PHI or working under HIPAA
• SQL for querying call and outcome data
• Twilio or telephony basics such as SIP, DTMF, and call routing
• Early-stage startup experience

Pay range

$112,090 - $160,330 USD