AI QA Engineer – Automation for Safe Clinical Releases

present of AI; United Kingdom; Remote; GBP 60,000 - 90,000; Full time

Job summary

Lyrebird Health seeks a hands-on QA engineer focused on automated testing for an AI clinical software product. You will write test code in TypeScript and Python, ensuring fast and reliable Playwright and API test suites, and you will make evidence-based release decisions.

The role emphasizes evaluating model outputs for clinical trust, using tools like DeepEval and Langfuse traces, and designing robust evaluations for non-deterministic responses. Experience with CI pipelines and Docker is valued.

Qualifications

• Hands-on testing focus for automated AI software.
• Write test code in TypeScript and Python.
• Own the final release decision based on evidence.

Responsibilities

• Develop and maintain automated test suites in TS/Python.
• Keep Playwright and API tests fast and reliable.
• Evaluate AI model outputs for clinical safety and trust.

Skills

TS testing

Python testing

CI pipelines

Docker

OpenAPI

Tools

Playwright

DeepEval

Langfuse traces

Job description

Lyrebird Health seeks a hands-on QA engineer focused on automated testing for an AI clinical software product. You will write test code in TypeScript and Python, ensuring fast and reliable Playwright and API test suites, and you will make evidence-based release decisions.

The role emphasizes evaluating model outputs for clinical trust, using tools like DeepEval and Langfuse traces, and designing robust evaluations for non-deterministic responses. Experience with CI pipelines and Docker is valued.