Senior AI QA Engineer for Government Platform

Scrumconnect Limited; Bridgend; Sur place; GBP 65 000 - 95 000; Plein temps

Résumé du poste

Scrumconnect Consulting in the United Kingdom seeks a Senior Quality Assurance Tester to lead testing for a Generative AI platform for a UK government department. You will shape test strategy across functional, non-functional, and AI-specific areas, including evaluation of AI outputs and RAG pipelines.

Responsibilities include API/ integration testing, CI/CD automation, and governance compliance with GOV.UK standards.

Qualifications

• Experience in a modern programming language (Java or Python) with coding standards.
• Experience in manual API testing and tools.
• Experience building API automation testing, preferably REST Assured.
• Adaptable team player with shift-left CI/CD experience (GitLab, Jenkins).
• Experience with non-functional testing (performance, accessibility).
• Agile environment exposure with multiple stakeholders.
• Comfort in secure, regulated environments handling sensitive data.
• Experience evaluating AI output quality (hallucination detection, relevance scoring).
• Experience delivering digital services in UK public sector.
• Familiarity with GOV.UK Service Standard expectations.

Responsabilités

• Define and own the test strategy across functional, non-functional and AI dimensions.
• Design and run structured evaluations of LLM outputs and track regressions.
• Perform manual and automated API testing across RAG pipelines and systems.
• Build and maintain automated tests in CI/CD pipelines (GitLab/Jenkins).
• Conduct non-functional testing (performance, accessibility) to ensure robustness.
• Verify guardrails and human-in-the-loop checkpoints per policy.
• Collaborate with Product, Engineering and Policy stakeholders in sprints.
• Ensure testing evidence aligns with GOV.UK Service Standards.
• Contribute test plans and evaluation results to Docs-as-Code.

Connaissances

Java

Python

API testing

CI/CD

AI/LLM evaluation

Agile / Scrum

Security & regulation awareness

Open-source LLM tooling

Cloud familiarity (AWS/Azure/GCP)

Shift-left testing

Outils

REST Assured

GitLab

Jenkins

k6

Gatling

JMeter

Description du poste

Scrumconnect Consulting in the United Kingdom seeks a Senior Quality Assurance Tester to lead testing for a Generative AI platform for a UK government department. You will shape test strategy across functional, non-functional, and AI-specific areas, including evaluation of AI outputs and RAG pipelines.

Responsibilities include API/ integration testing, CI/CD automation, and governance compliance with GOV.UK standards.