Senior AI Quality Engineer — Evaluation & Test Automation

Xapien; Greater London; On-site; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Private health insurance

Life Insurance

Unlimited holidays

£1k professional development fund per

Employee equity program

Job summary

Xapien in London is expanding its AI‑driven due diligence platform and seeks a senior test automation engineer to strengthen our evaluation systems and CI workflows.

You will automate tests, improve datasets, and help ensure non‑deterministic outputs are reliable across multi‑tenant environments, with a focus on quality and speed.

Join a small, fast‑moving team and contribute to real‑world risk intelligence used by law firms and financial institutions.

Qualifications

• Proven experience in test automation engineering.
• Strong Python and modern test stacks.
• Experience with Playwright/pytest and CI/CD.

Responsibilities

• Enhance and run layered eval harnesses with automated checks and LLM scoring.
• Maintain versioned golden datasets for reproducibility and auditing.
• Track signals like groundedness and hallucination rates in investigative output.
• Run and improve a tiered CI model across multiple personas and browsers.
• Investigate flaky tests and fix root causes.

Skills

Test automation

Python

CI/CD

Playwright

pytest

Tools

Playwright

pytest

CI/CD pipelines

Job description

Xapien in London is expanding its AI‑driven due diligence platform and seeks a senior test automation engineer to strengthen our evaluation systems and CI workflows.

You will automate tests, improve datasets, and help ensure non‑deterministic outputs are reliable across multi‑tenant environments, with a focus on quality and speed.

Join a small, fast‑moving team and contribute to real‑world risk intelligence used by law firms and financial institutions.