Senior QA Engineer

Embrace Software Inc.; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Comprehensive Health Benefits

Flexible Remote Work

Professional Growth Support

Job summary

Embrace Healthcare Group seeks a Senior QA Engineer to build EPOWERdoc's AI-native quality engineering practice. This hands-on role involves selecting and architecting the test automation stack, writing code, and guiding AI-driven test design. You will ensure safety, accuracy, and defensible testing for clinical workflows.

You will partner with engineers, product, and clinical content experts to test AI features and maintain robust quality across the platform.

Qualifications

• 5+ years of QA or software testing, with at least 3 years building test automation.
• Experience in SaaS or web applications and risk-based testing concepts.
• Hands-on with modern automation tools; Playwright preferred.

Responsibilities

• LeadQA strategy and define test plans, coverage goals, and quality gates.
• Architect the QA tech stack for UI, API, performance, accessibility and security testing.
• Build scalable frameworks; replace manual regression with automated coverage.
• Design load, stress, and soak tests; establish performance baselines.
• Integrate automated tests into CI/CD; enable parallel execution and AI-assisted analysis.
• Drive defect management and root-cause analysis throughout lifecycle.
• Lead exploratory testing of new features and complex clinical workflows.
• Collaborate with developers, product managers, designers, and clinicians.

Skills

QA engineering

Test automation

Playwright

API testing

CI/CD

Performance testing

Locust/k6

SaaS/web apps

Education

Bachelor's degree in Computer Science or Software Engineering

Tools

Playwright MCP server

Locust

k6

OWASP ZAP

Pact

Job description

We're rebuilding vertical software with AI — from the inside. Embrace owns the software running inside 16% of the Fortune 500, 45+ state agencies, and 450+ banks and credit unions. We acquire entréched vertical software businesses and rebuild them around AI — products, operations, go-to-market, all of it. Our Venture Lab launches new AI-native products into those same markets, using the distribution and customer relationships our portfolio already owns. You'll ship AI into production against real workflows, real customers, and a P&L you can see move within a quarter. We hire people who want scope, speed, and ownership, and who are tired of working on AI that never reaches a customer. If you want to spend the next five years shipping into software that already runs the economy, talk to us.

Job Description

This is a remote position.

Embrace Healthcare Group powers some of the most innovative healthcare technology solutions in North America, with a focus on enhancing clinical efficiency and compliance across emergency care settings.

EPOWERdoc, a leading SaaS platform within the Embrace Healthcare Group portfolio, specializes in Emergency Department Information Systems (EDIS) tailored for hospitals of all sizes. Designed by clinicians for clinicians, EPOWERdoc streamlines emergency room documentation with intuitive workflows, customizable templates, peer-reviewed clinical content, and real-time coding assistance—helping providers reduce documentation time and focus more on patient care.

We're looking for a Senior QA Engineer to build EPOWERdoc's next-generation quality engineering practice, AI-native from day one. This is a hands‑on builder role: you'll choose and architect our test automation stack, write much of the code yourself, and use AI coding agents every day to design, generate, maintain, and triage tests.

You'll also own how we test AI itself. You'll design the evaluation harnesses and guardrails that show EPOWERdoc's AI-powered capabilities are accurate, safe, and dependable enough for an emergency department. You'll partner closely with developers, product managers, designers, and clinical content experts.

Key Responsibilities:

• Lead QA strategy: Own the end-to-end quality strategy for EPOWERdoc, defining test plans, risk-based coverage goals, and quality gates for every release.
• Architect the QA tech stack: Evaluate, select, and document the tooling for UI, API, performance, accessibility, and security testing, recording each decision and its trade-offs in architecture decision records.
• Build automation and expand coverage: Design and maintain scalable test frameworks for web applications, APIs, and integrations. Set standards clear enough that engineers and AI agents can extend them reliably. Use these frameworks to replace manual regression with automated coverage of critical clinical workflows such as triage, documentation, orders, and discharge.
• Performance and reliability: Build load, stress, and soak tests that model real emergency department peak volumes, establish performance baselines, and catch regressions before they reach production.
• Own CI/CD quality: Integrate automated tests into CI/CD pipelines so every pull request and build is validated quickly. Use parallel execution and AI‑assisted test impact analysis to keep feedback loops short as the suite grows.
• Defect management: Work with engineering teams to identify, document, prioritise, and resolve defects early in the development lifecycle, and drive root‑cause analysis.
• Exploratory testing: Lead exploratory testing of new features and complex clinical user flows, combining human judgement with AI‑driven exploration where automation alone adds less value.
• Agile partnership: Bring a quality perspective to sprint planning, refinement, and retrospectives, including estimates, acceptance criteria, and release readiness.
• Quality advocacy: Champion a quality‑first culture from code reviews to release management. Report on quality metrics and trends such as escaped defects, flakiness, and test cycle time.
• Security and compliance: Ensure testing practices, test data, and the use of AI tools meet HIPAA and healthcare data security standards, with no real PHI in test environments or AI prompts.

AI‑NATIVE QUALITY ENGINEERING

AI is part of how this team works every day. In this role you will:

• AI-assisted test design: Use LLMs to turn user stories, acceptance criteria, and clinical specifications into test scenarios, edge cases, and boundary conditions, then review and sharpen them with sound test‑design judgement.
• Agentic test authoring: Use AI coding agents and Playwright’s MCP server and test agents (planner, generator, and healer) to create and maintain automated tests. Hold generated code to the same review and quality bar as production code.
• Self‑healing and failure triage: Apply AI to locator healing, flaky‑test detection, and automated failure analysis: clustering failures, summarising likely root causes, and routing defects to the right owner.
• Synthetic test data: Make realistic, PHI‑free synthetic patient records and clinical scenarios the default for functional, performance, and AI evaluation testing, using tools such as Synthea or LLM‑driven generators.
• Testing AI features: Design evaluation frameworks for AI‑powered EPOWERdoc capabilities. These include golden datasets, scoring rubrics, LLM‑as‑judge grading calibrated against human review, and regression gates that run whenever a model, prompt, or knowledge source changes.
• AI safety and guardrails: Test AI features for hallucination, clinical and coding accuracy, bias, PHI leakage, prompt injection, and graceful failure. Work with clinical experts to define what “good enough for the ED” means in measurable terms.
• Measure and share impact: Track how AI changes test authoring time and maintenance effort, and share what works with other Embrace engineering teams.

THE STACK YOU’LL SHAPE

This is our starting point, not a mandate. You'll own the final decisions and the reasoning behind them.

• UI and end‑to‑end: Playwright with TypeScript, including visual regression and automated accessibility checks (axe‑core).
• API and integration: Playwright API testing, contract testing (for example Pact), and validation of HL7 and FHIR interfaces.
• Performance: Locust or k6 for load, stress, and soak testing.
• Security: OWASP ZAP and dependency scanning in CI, plus the OWASP Top 10 for LLM Applications as the baseline for AI features.
• AI tooling: AI coding agents (such as Claude Code, GitHub Copilot, or Cursor), Playwright MCP and test agents, and LLM evaluation frameworks such as Promptfoo, DeepEval, or Ragas, with tracing through Langfuse or LangSmith.
• CI/CD and reporting: Tests running on every pull request, with Allure or a similar dashboard for trends, flakiness, and release readiness.

WHAT SUCCESS LOOKS LIKE

• First 30 days: You understand EPOWERdoc’s clinical workflows, existing test assets, and release process. You've proposed the target QA stack and AI-native approach as written architecture decision records.
• By 90 days: Playwright smoke and regression suites cover the most critical clinical workflows and run on every pull request. A performance baseline exists for peak load. AI-assisted test authoring is in daily use with measured results.
• By 6 months: Manual regression effort is a small fraction of today's. Flaky tests are tracked and trending down. An evaluation harness gates every release of AI-powered features.

Requirements

• Experience: 5+ years of professional QA or software testing experience, including at least 3 years building test automation, ideally in a SaaS or web application environment.
• QA foundations: Solid understanding of QA methodologies, risk-based testing, test design techniques, and software development life cycles.
• Test automation: Hands‑on experience with modern automation tools, with Playwright strongly preferred (Cypress or Selenium also considered). This includes building scalable frameworks from the ground up, managing test data, keeping suites stable and fast, and a record of well‑reasoned tooling decisions.
• AI-native practice: Daily use of AI coding agents in test engineering, with examples you can walk us through, and the judgement to spot when AI output is wrong or shallow.
• Testing AI systems: Understanding of how LLM features fail and how to test non‑deterministic outputs with eval datasets, rubric or model‑based grading, and pass thresholds.
• Performance testing: Hands‑on experience with Locust, k6, JMeter, or similar tools, including designing realistic workload models and interpreting the results.
• API and CI/CD: Solid experience with API testing, CI/CD tools, and Git, and with running automated suites within build pipelines.
• Communication and ownership: Ability to explain quality risks clearly to engineers, product managers, and leadership, with strong analytical skills and a proactive, self‑starter attitude.
• Ways of working: Experience working in a highly remote, fast‑paced, and agile environment across time zones.
• Education: Bachelor's degree in Computer Science, Software Engineering, or a related field (or equivalent experience).

Nice to have

• Healthcare experience: Work in healthcare or health tech, especially EHR, EDIS, or other regulated clinical software (for example, ONC Health IT certification testing). This includes familiarity with HL7 v2, FHIR, and HIPAA requirements for test data and environments.
• AI tooling experience: Hands‑on experience with LLM evaluation and observability tools, or building MCP servers and custom AI agents for engineering teams.
• Security and accessibility testing: Experience testing web and AI applications, including prompt‑injection testing, red‑teaming, and WCAG.
• Competitive salary aligned with UK working hours, ensuring fair and transparent compensation.
• Global Team Collaboration—work alongside extremely talented professionals from around the world.
• Professional Growth Support with access to training programs designed to enhance your skills and expand your knowledge.
• Clear Career Progression and excellent opportunities for long‑term advancement within the organization.
• Comprehensive Health Benefits to support your wellbeing and peace of mind.
• Flexible Remote Work Options that empower you to balance work and life effectively.