Sr. Quality Automation Engineer

Authenticx; Indianapolis (IN); Hybrid; USD 120,000 - 150,000; Full time

Job summary

Authenticx is seeking a Senior Quality Automation Engineer in Indianapolis (Preferred) or United States-Remote to design, build, and maintain automated tests across our healthcare conversation intelligence platform. You will translate customer workflows and product requirements into automated coverage, owning automation for critical areas and prioritizing based on impact and risk.

You will partner with Product, Engineering, Data Science, Implementation, and Customer teams to improve testability,

Qualifications

• Experience testing LLM or ML features using representative datasets, expected outcomes, and threshold based regression checks.
• Healthcare, contact center, or regulated environment experience.
• Ingestion pipelines, event driven systems, and third party integrations.

Responsibilities

• Build Automation Around Real Customer Use Cases.
• Design, write, maintain, and troubleshoot automated tests for UI workflows, APIs, backend services, data pipelines, integrations, reporting, and AI-powered features.
• Use Playwright and Bruno/Postman to automate critical user journeys and API validation.
• Integrate automated tests into CI/CD pipelines and investigate flaky tests.

Skills

LLM/ML testing experience

Automation testing

Test design & automation frameworks

Tools

C#/.NET

PostgreSQL

Azure

Job description

Indianapolis, IN (Preferred) or United States-Remote

Senior Quality Automation Engineer

About Authenticx
Authenticx is a fast-growing, venture-backed, enterprise AI company transforming healthcare through conversation intelligence. We help healthcare organizations uncover insights hidden in millions of customer interactions to improve patient, member, and business outcomes.

Role Summary
The Senior Quality Automation Engineer is a hands-on role responsible for designing, building, maintaining, and integrating automated tests across Authenticx's healthcare conversation intelligence platform. You will translate customer workflows, product requirements, user personas, and healthcare realities into automated coverage that protects the experiences that matter most. You will own automation for critical product areas and prioritize coverage based on customer impact, product risk, release frequency, and business value.
You will partner with Product, Engineering, Data Science, Implementation, and Customer teams to improve product quality, testability, and release confidence. We are looking for someone who brings strong technical judgment, product sense, curiosity, and ownership to quality engineering in a fast-paced startup environment.

What You'll Do

Build Automation Around Real Customer Use Cases

• Learn how Authenticx customers use the platform across healthcare quality, coaching, compliance, operations, analytics, and customer experience workflows.
• Translate workflows, acceptance criteria, production issues, and known risk areas into meaningful automated test scenarios.
• Determine what should be automated, which testing layer is most appropriate, and the level of coverage needed.
• Balance UI, API, integration, and data-level testing to create reliable and maintainable coverage.

Build and Maintain Automated Coverage

• Design, write, maintain, and troubleshoot automated tests for UI workflows, APIs, backend services, data pipelines, integrations, reporting, and AI-powered features.
• Build and maintain smoke, regression, acceptance, integration, and end-to-end test suites.
• Use Playwright to automate critical user journeys and product workflows.
• Use Bruno/Postman to validate APIs, payloads, integrations, authentication, error handling, and backend behavior.
• Use SQL to validate data accuracy, workflow integrity, processing outcomes, reporting outputs, and customer-facing analytics.
• Build reusable automation utilities, fixtures, helpers, and test data patterns.
• Integrate automated tests into CI/CD pipelines to provide earlier and more reliable quality signals.
• Investigate flaky tests, environmental issues, timing problems, and test-data failures, and distinguish them from true product defects.
• Identify high-risk coverage gaps and build automation to address them.

Partner Cross-Functionally to Improve Quality

• Work with Product, Engineering, Data Science, Implementation, and Customer teams to understand requirements, risks, and real-world usage.
• Partner with developers to improve testability, observability, logging, test data setup, and API contracts.
• Use logs, monitoring tools, database queries, API responses, and product data to investigate failures and validate system behavior.
• Communicate defects, automation gaps, and release risks clearly.
• Share automation patterns and testing practices that strengthen the broader quality engineering discipline.

Validate AI-Powered Product Experiences

• Build automated validation for classifications, model outputs, prompt and configuration changes, structured responses, extraction workflows, and downstream behavior.
• Develop repeatable test scenarios for deterministic and semi-deterministic AI behaviors.
• Validate expected outputs, required fields, edge cases, failure conditions, and downstream workflow impacts.
• Use representative test data and known expected outcomes to detect regressions.
• Partner with Data Science and Engineering to improve the testability and regression coverage of AI-powered features.

Preferred Qualifications

• C#/.NET, PostgreSQL, and Azure
• Experience testing LLM or ML features using representative datasets, expected outcomes, and threshold based regression checks
• Ingestion pipelines, event driven systems, and third party integrations
• Healthcare, contact center, or regulated environment experience
• AI assisted development tools for test creation and debugging

U.S. Standard Demographic Questions

We invite applicants to share their demographic background. If you choose to complete this survey, your responses may be used to identify areas of improvement in our hiring process.

Voluntary Self-Identification

For government reporting purposes, we ask candidates to respond to the below self-identification survey.Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiringprocess or thereafter. Any information that you do provide will be recorded and maintained in aconfidential file.

As set forth in Authenticx’s Equal Employment Opportunity policy,we do not discriminate on the basis of any protected group status under any applicable law.

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection.As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measurethe effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categoriesis as follows:

A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.

A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.