Senior QA Automation Engineer - AI-Driven Web Testing

Evolent; United States; On-site; USD 115,000 - 120,000; Full time

Benefits offered by this job

Health benefits

Autonomy and flexible work schedule

Job summary

Evolent is seeking a Senior QA Automation Engineer to lead the quality strategy for the Evolent Platform web UX. This role designs and maintains end-to-end, component, accessibility, contract, and test automation for a React and Next.js application and owns the quality signals that determine release readiness.

This individual works independently with minimal supervision, applies creativity to testing challenges, sets and upholds the automation patterns others follow, and mentors quality

Qualifications

• 5+ years of experience in software quality assurance, including 3+ years of hands-on test automation for modern web applications.
• Strong hands-on experience with Playwright using TypeScript or JavaScript for end-to-end automation.
• Experience with component testing using Vitest or Jest and React Testing Library.
• Experience testing React-based web applications, including asynchronous and streaming behavior.
• Hands-on accessibility testing experience with axe-core and manual screen readers (JAWS, NVDA, VoiceOver) against WCAG 2.1/2.2 AA and Section 508.
• Experience with API testing and contract testing, including schema-based validation of APIs or event streams.
• Experience with load or performance testing tools (e.g., k6, JMeter).
• Experience creating and maintaining automation jobs in CI/CD tools (e.g., GitHub Actions with YAML) and version control with Git.
• Knowledge of test management and defect tracking tools (e.g., JIRA, ALM).
• In-depth knowledge of testing methodologies, including functional, regression, accessibility, and performance testing.
• Experience working in Agile development environments.
• Strong problem-solving skills and a proactive approach to troubleshooting and issue resolution.
• Excellent communication and collaboration skills, with the ability to mentor others.

Responsibilities

• Develop, implement, and manage comprehensive test strategies for the user experience, covering functional, regression, accessibility, performance, and AI-output quality.
• Participate in code reviews, design discussions, and product planning to ensure quality is built into all phases of the software lifecycle.
• Define and uphold test automation patterns and standards in partnership with testing automation architects and front-end leads.
• Mentor quality engineers, promoting best practices in testing, automation, and quality assurance.
• Drive continuous improvement initiatives in testing processes, tools, and methodologies, and evaluate emerging testing technologies for applicability.
• Lead the creation, maintenance, and optimization of Playwright end-to-end suites and Vitest component tests for React and Next.js features.
• Build contract tests for the AG-UI event stream, verifying event sequencing, shared state, tool-call events, and human-in-the-loop approval interrupts.
• Build API tests for backend-for-frontend endpoints and .NET and Python (FastAPI) services.
• Test streaming behavior and resilience, including slow, interrupted, and failed agent calls, retries, and timeouts.
• Plan and execute basic load and performance tests for streaming endpoints and data-heavy screens.
• Integrate and execute automated suites in GitHub Actions, maintaining a consistently high pass rate and reliable results.
• Lead accessibility testing against WCAG 2.1/2.2 AA and Section 508, combining automated axe-core checks with manual screen reader testing using JAWS, NVDA, and VoiceOver.
• Partner with engineering and design to triage accessibility defects and support independent accessibility conformance reviews.
• Isolate, analyze, document, and prioritize defects before escalating to developers, and collaborate with developers, business analysts, and stakeholders for timely resolution.
• Analyze test and evaluation results, generate quality metrics, and report on quality state and release readiness to stakeholders.
• Present work in Agile ceremonies and lead technical knowledge-sharing sessions.
• Utilize approved AI-assisted engineering tools to accelerate test design, test script development, test data generation, documentation, troubleshooting, and defect analysis.
• Evaluate and validate AI-generated tests, scripts, and analysis for correctness, security, maintainability, compliance, and business appropriateness.
• Continuously identify opportunities to improve testing productivity and solution quality through AI-enabled workflows and automation.
• Contribute reusable prompts, skills, agents, testing playbooks, and AI-assisted testing patterns to benefit the broader engineering organization.
• Follow enterprise policies governing AI usage, intellectual property protection, security, privacy, PHI handling, and Responsible AI practices.
• Ensure no confidential, regulated, or protected information (including PHI in test data) is used with unapproved AI services or tooling; use synthetic data for testing.
• Participate in peer reviews and validation processes for AI-assisted solutions.

Skills

QA Automation

Playwright

React Testing Library

CI/CD

Mentoring

Education

Bachelor's degree in CS/Software Eng

Tools

GitHub Actions

Jest

axe-core

JAWS

NVDA

Job description

Evolent is seeking a Senior QA Automation Engineer to lead the quality strategy for the Evolent Platform web UX. This role designs and maintains end-to-end, component, accessibility, contract, and test automation for a React and Next.js application and owns the quality signals that determine release readiness.

This individual works independently with minimal supervision, applies creativity to testing challenges, sets and upholds the automation patterns others follow, and mentors quality