QA Engineer, Voice AI Agents — Remote

arbiter-ai; United States; Remote; USD 180,000 - 200,000; Full time

Benefits offered by this job

Equity

Generous PTO

Health Insurance

Dental & Vision

Life Insurance

Disability Insurance

IRA Matching

Development Budget

Wellness Programs

Job summary

Arbiter is the AI-powered care orchestration platform building Agentic workflows in healthcare. We’re hiring a QA Engineer to dig into calls, transcripts, and campaigns, tracing root causes with engineers and deployment teams.

This role is remote or onsite in our New York City or Boca Raton offices, offering exposure to high-scale healthcare tech, equity, and a culture focused on excellence and growth. Benefit from a comprehensive package, including competitive salary, PTO, health/dental/vision

Qualifications

• 3 to 5 years in QA, support engineering, technical operations, or a similar role on a production software platform.
• A strong debugging instinct. You like finding out why something broke, and you don't stop at the first plausible answer.
• Comfort reading logs, querying data with SQL, and working with APIs and JSON or YAML configuration.
• Experience writing test plans and bug reports that engineers trust.
• Clear written communication and precise documentation habits.
• Good judgment on severity and prioritization when several issues are live at once.

Responsibilities

• Investigate reported and detected issues on live campaigns, including dropped calls, wrong agent responses, failed handoffs, incorrect scheduling outcomes, and data mismatches.
• Trace problems across call recordings, transcripts, system logs, campaign configuration, and integration data to find the actual root cause.
• Write clear, reproducible bug reports that engineers can act on right away, with severity, member impact, and supporting evidence.
• Triage incoming issues, separating agent behavior from configuration errors, data problems, and platform bugs, and route each to the right owner.
• Run pre-launch QA on new campaigns and workflow changes: test scripts, edge cases, escalation paths, and handoffs to nurses or live staff.
• Carry out ongoing conversation QA on live campaigns against defined quality standards, and flag trends before they turn into customer issues.
• Check that campaigns behave as designed: correct targeting, sequencing, disposition logic, and outcome capture.
• Help build and maintain the QA framework: test cases, scoring rubrics, regression suites, and release checklists.
• Automate repetitive checks where it makes sense, such as transcript analysis, config validation, and outcome reconciliation.
• Track quality metrics and report patterns to deployment, product, and engineering so issues get fixed at the source.

Skills

QA experience

SQL

APIs

Log analysis

Python scripting

Communication

Tools

Observability tools

Call analytics tools

Job description

Arbiter is the AI-powered care orchestration platform building Agentic workflows in healthcare. We’re hiring a QA Engineer to dig into calls, transcripts, and campaigns, tracing root causes with engineers and deployment teams.

This role is remote or onsite in our New York City or Boca Raton offices, offering exposure to high-scale healthcare tech, equity, and a culture focused on excellence and growth. Benefit from a comprehensive package, including competitive salary, PTO, health/dental/vision