Quality Assurance Engineer (Automation)

Lyrebird Health; United Kingdom; On-site; GBP 55,000 - 75,000; Full time

Job summary

Lyrebird Health seeks a Quality Assurance Engineer (Automation) to own the platform's quality from commit to release. You’ll develop fast, reliable Playwright suites and contract tests that guard against backend changes, while evaluating AI outputs for clinical safety.

You’ll collaborate with SRE and engineering to ensure robust, production-like test code. You’ll switch between TypeScript and Python, cutting flaky tests and accelerating pipelines, with an emphasis on measurable signals and

Qualifications

• Strong test automation experience in TypeScript and Python.
• Hands-on Playwright experience with component testing.
• Solid pytest skills including async tests, fixtures, mocking and data factories.
• Experience running tests in CI pipelines (Buildkite) and in Docker.
• Working knowledge of SQL and PostgreSQL and testing at the database layer.
• Experience with API contract testing against OpenAPI schemas.

Responsibilities

• Optimise Playwright end-to-end suite to reduce run time and flaky tests.
• Extend API and partner integration suites and build contract tests from OpenAPI schemas.
• Add coverage gates to Buildkite pipelines and report quality by product area.
• Support CI/CD development and oversight with engineering teams.
• Test clinical model outputs using DeepEval, openevals and Langfuse traces.
• Run Locust load tests ahead of major releases and report findings to SRE.
• Build signals for SRE: health checks, synthetic monitors and post-deployment tests.
• Own the release quality gate with evidence-backed decisions.
• Raise testing standards across engineering through patterns, fixtures and code reviews.

Skills

TypeScript

Python

Playwright

Vitest

Testing Library

pytest

CI/CD

Docker

SQL

OpenAPI

API contract testing

Tools

Buildkite

OpenAPI

PostgreSQL

Locust

Job description

Quality Assurance Engineer (Automation)

The Role

You’ll own the quality of the Lyrebird platform, from the first commit to the release gate. This is a hands‑on, code‑first engineering role, not manual test scripting. You’ll make our Playwright end‑to‑end and integration suites faster and more reliable, build contract tests that stop backend changes from silently breaking the web app, and design ways to evaluate clinical model output where exact‑match assertions don’t work. You’ll also be the quality partner to our SRE function, giving them the health checks, synthetic monitors, load test results and post‑deployment signals they need. When a release is ready to go, you’ll give a clear yes or no, backed by evidence.

This role suits an engineer who sees testing as a craft and treats a test suite as production code. You’re comfortable moving between TypeScript and Python in the same day, you get real satisfaction from deleting a flaky test and cutting minutes off a pipeline, and you’d rather help other engineers write better tests than be the only person who writes them. You’ll enjoy the hard problem at the centre of this role: how to prove that non‑deterministic AI output is safe and accurate enough for clinicians to rely on.

About Us

Lyrebird Health is transforming the quality and accessibility of healthcare by automating clinicians’ most time‑consuming tasks.

Thousands of clinicians across multiple disciplines use Lyrebird every day in environments where accuracy, safety, and time matter deeply. Everything we build is grounded in earning trust, reducing cognitive load, and delivering meaningful impact at scale.

What you’ll do

• Optimise our Playwright end‑to‑end suite, cutting run time and removing flaky tests for good

• Extend our API and partner integration suites, and build contract tests from our OpenAPI schemas so no backend change can break the web app without warning

• Add coverage gates to our Buildkite pipeline, using reporting that already breaks quality down by product area

• Support and collaborate with our engineering teams with CICD development and oversight

• Test the clinical output of our models with DeepEval, openevals and Langfuse traces, designing evaluations that hold up when responses change from run to run

• Run load tests with Locust ahead of major releases and hand clear findings to the SRE function

• Build the signals SRE depends on: health checks, synthetic monitors and post‑deployment tests

• Own the release quality gate, giving a clear yes or no with the evidence behind it

• Raise the standard of testing across engineering through shared patterns, fixtures and thoughtful code review

What you’ll bring

• Strong test automation experience in both TypeScript and Python, writing tests as production‑quality code

• Deep hands‑on experience with Playwright, plus component testing with Vitest and Testing Library

• Solid pytest skills, including async tests, fixtures, mocking and test data factories such as polyfactory or Faker

• Experience running tests in CI pipelines (ideally Buildkite) and in Docker, with coverage measurement and reporting

• Working knowledge of SQL and Postgres, and confidence testing at the database layer

• Experience with API contract testing against OpenAPI schemas

Nice to have

• Experience evaluating large language model output with tools such as DeepEval, openevals or Langfuse

• Load testing experience with Locust

• Familiarity with Svelte, Storybook, or Supabase and its CLI‑based database testing

• Experience with observability tools such as Sentry, OpenTelemetry and Honeycomb

• A background in healthcare, clinical software or other regulated, safety‑critical environments

Every test you write protects a clinician who is relying on Lyrebird in the middle of a busy day, and the patient sitting in front of them. When our platform is fast, stable and trustworthy, clinicians spend less time on admin and more time on care. You’ll be the person who makes sure we can promise that with confidence, release after release.

We’re building a team that reflects the diversity of the people who benefit from our work. We want Lyrebird to be a place where everyone feels safe, supported, and able to thrive. If you’re from an underrepresented background in tech, we strongly encourage you to apply, even if you don’t meet every requirement.