AI-Powered Quality Engineer & Test Automation

Baylor Scott & White Health; Providence (RI); On-site; USD 110,000 - 160,000; Full time

Benefits offered by this job

Immediate eligibility for health and w

401(k) savings plan with dollar-for-dl

Tuition Reimbursement

PTO accrual beginning Day 1

Job summary

Baylor Scott & White Health is seeking a Quality Engineer, AI & Test Automation to modernize quality practices via automation-first, engineering-integrated, and AI-enabled approaches. You will design, build, and maintain automated test suites for APIs, UI, and data validation across digital products and AI use cases.

You will partner with Product, Engineering, AI Foundation, Security, Privacy, and Clinical teams to embed quality throughout the lifecycle and ensure release readiness with robust

Qualifications

• 5–8+ years of experience in quality engineering, software testing, test automation, SDET, software engineering, AI quality, or related technology delivery roles.
• Hands-on experience designing and implementing automated tests for APIs, web or mobile applications, workflows, data validation, integrations, or end-to-end product experiences.
• Experience working with engineering teams in agile, DevSecOps, CI/CD, or modern product-delivery environments.
• Exposure to AI-enabled products, machine learning systems, large language models, conversational experiences, agentic workflows, automation, or decision-support technologies preferred.
• Experience with defect triage, root-cause analysis, test evidence documentation, release readiness, and quality metrics.

Responsibilities

• Develop test strategies, test plans, acceptance criteria, and quality approaches in partnership with stakeholders.
• Translate requirements, user stories, workflows, APIs, data dependencies into test scenarios and quality validation plans.
• Identify quality risks early and recommend validation approaches including automated, manual, exploratory, integration, regression, performance, accessibility, or AI-specific methods.
• Provide release-readiness input based on test evidence, defects, and quality metrics.
• Design, build, maintain, and improve automated test suites for API, UI, integration, end-to-end, regression, data, and workflow validation.
• Integrate automated tests into CI/CD pipelines and delivery workflows to surface quality signals earlier.
• Create reusable test data, utilities, fixtures, scripts, and automation patterns across teams.
• Improve test reliability, execution time, coverage, and signal-to-noise by reducing brittle scripts.
• Support testing of AI-enabled experiences, including agent workflows, generated responses, retrieval behavior, tool/API calls, escalation paths, guardrails, and human-in-the-loop patterns.
• Define evaluation datasets, expected behaviors, failure modes, scoring rubrics, regression scenarios, and quality thresholds for AI-enabled capabilities.
• Use AI-enabled testing tools to generate test cases, analyze logs, triage defects, identify gaps, summarize results, and accelerate quality workflows.
• Partner with AI Quality, Architecture, Engineering, and Product teams to validate usefulness, reliability, and alignment to quality expectations.
• Identify, document, reproduce, triage, and communicate defects with technical detail; analyze production feedback to identify root causes and improvements.
• Collaborate with teams to improve testability, observability, logging, error handling, and diagnostics.
• Contribute to retrospectives and quality reviews to reduce rework and improve release confidence.
• Support post-release monitoring and quality reporting.

Skills

Quality testing concepts

CI/CD integration

AI-enabled testing

Analytical communication

Defect triage

Education

Bachelor’s degree in Computer Science, Engineering, Information Systems, Data Science, or related field

Tools

Playwright

Cypress

Selenium

Appium

PyTest

Postman/Newman

JUnit

TestNG

Job description

Baylor Scott & White Health is seeking a Quality Engineer, AI & Test Automation to modernize quality practices via automation-first, engineering-integrated, and AI-enabled approaches. You will design, build, and maintain automated test suites for APIs, UI, and data validation across digital products and AI use cases.

You will partner with Product, Engineering, AI Foundation, Security, Privacy, and Clinical teams to embed quality throughout the lifecycle and ensure release readiness with robust