Senior AI QA Lead for Government Platforms

Scrumconnect Limited; Newcastle upon Tyne; Presencial; GBP 60.000 - 85.000; Jornada completa

Descripción de la vacante

Scrumconnect Consulting in Newcastle, United Kingdom, is seeking an experienced Senior Quality Assurance Tester to lead testing strategy for a government AI platform. Responsibilities include evaluating AI output quality, API testing, and CI/CD integration within agile teams.

You will design LLM evaluations, oversee non-functional testing, and ensure governance compliance across the delivery lifecycle, working in secure government environments.

Formación

• Experience in a modern programming language (preferably Java or Python).
• Experience in manual API testing and tools.
• Experience building and maintaining API automation testing, preferably with REST Assured or similar tools.
• Adaptable team player with shift-left principles and CI/CD tools like GitLab/Jenkins.
• Experience conducting non-functional testing (performance and accessibility).
• Experience working in an agile environment with multiple stakeholders.
• Comfortable in secure, regulated environments handling sensitive data.
• Experience evaluating AI output quality (hallucination, relevance, retrieval).
• Experience delivering digital services in UK central government or public sector.
• Familiarity with GOV.UK Service Standard expectations and governance.
• Exposure to open-source LLM tooling or evaluation frameworks.
• Familiarity with AI/ML tooling on AWS, Azure or GCP.

Responsabilidades

• Define and own the test strategy across functional, non-functional and AI-specific quality dimensions (answer quality, retrieval accuracy, hallucination rate).
• Design and run structured evaluations of LLM outputs with engineering to track quality and flag regressions.
• Carry out manual and automated API testing across application logic connecting RAG pipelines to systems.
• Build and maintain automated test suites integrated into CI/CD pipelines.
• Perform performance testing and accessibility testing to ensure robustness and inclusivity.
• Verify guardrails and human-in-the-loop checkpoints work as intended.
• Collaborate with Product, Engineering and Policy stakeholders to embed testing in sprints.
• Ensure testing and release evidence aligns with GOV.UK standards and governance.
• Contribute test plans and evaluation results to Docs-as-Code artefacts for knowledge transfer.

Conocimientos

Programming (Java/Python)

Manual API testing

API automation testing (REST Assured)

Shift-left / CI/CD

CI/CD tools (GitLab, Jenkins)

Non-functional testing (performance, 1

Agile delivery teamwork

AI/Large Language Model evaluation

Security/regulatory environment

Open-source LLM tooling exposure

Cloud familiarity (AWS/Azure/GCP)

Herramientas

GitLab

Jenkins

REST Assured

k6

Gatling

JMeter

Descripción del empleo

Scrumconnect Consulting in Newcastle, United Kingdom, is seeking an experienced Senior Quality Assurance Tester to lead testing strategy for a government AI platform. Responsibilities include evaluating AI output quality, API testing, and CI/CD integration within agile teams.

You will design LLM evaluations, oversee non-functional testing, and ensure governance compliance across the delivery lifecycle, working in secure government environments.