Principal AI Quality Engineer

Fourth; Greater London; Hybrid; GBP 120,000 - 180,000; Full time

Benefits offered by this job

Holiday allowance

Hybrid work

Gym discounts

Company laptop

Healthcare costs coverage

Certifications support

Annual meet-ups

Parenting benefits

Cycle to work

Pension & life insurance

On-demand pay

Job summary

Fourth LTD is seeking a Principal AI Quality Engineer to define and drive our AI‑driven QA strategy from the front, reporting to the Global Head of Service Delivery. You will shape how quality is engineered across Fourth’s AI and software ecosystems, setting vision, standards, and governance for AI‑assisted and agentic development.

You will own quality KPIs, defect management, and cross‑functional collaboration with Product and Engineering to deliver reliable AI‑driven software at scale.

Qualifications

• Experience leading QA strategy for enterprise-scale AI/ML projects.
• Ability to define and govern quality frameworks for AI-driven systems.
• Proven track record of building and maturing test automation across multiple teams.
• Experience shaping QA for AI-assisted and agentic development.
• Strong definition and analysis of quality KPIs and gates.
• Ability to influence senior engineering and product stakeholders.
• Experience creating and maintaining technical strategy documentation.
• Expertise in Agile delivery at scale.
• Excellent written and spoken English communication.

Responsibilities

• Define, own, and evolve the AI-driven global QA strategy and roadmap.
• Establish QA tools, frameworks, standards, and ways of working globally.
• Drive unification and improvement of testing processes across delivery teams.
• Serve as the escalation point for complex testing and quality challenges.
• Champion modern testing technologies and AI adoption in QA.
• Define and own quality KPIs and quality gates for products.
• Own the defect management process and enterprise defect prevention.
• Lead QA for AI development end-to-end, including AI code generation and agentic AI.
• Establish governance for non-deterministic AI systems and testing strategies for LLM-based features.
• Set enterprise standards for validating AI-generated code.

Skills

AI QA strategy

QA at scale

Test automation expertise

AI development QA

Quality KPIs

Technical leadership

Strategy documentation

Agile at scale

Stakeholder management

Tools

Playwright

Jenkins

Azure DevOps

Octopus Deploy

Prometheus

Grafana

Sumo Logic

GitHub Copilot

Job description

Position Overview

This is an exciting opportunity to define how a global technology company assures quality in the age of AI. Fourth is seeking an exceptional, strategically minded Principal AI Quality Engineer to drive our AI‑driven global QA strategy from the front. Reporting to the Global Head of Service Delivery within Technology, you will be the most senior quality voice in the organisation, shaping how quality is engineered across Fourth’s systems in an era of AI‑assisted and agentic AI development. You will set the vision, standards, and frameworks that build quality, reliability, and trust into everything we ship, aligning the QA strategy directly with the company’s technical vision, roadmap, and business strategy. If you want to leave a lasting mark on how software quality and AI come together at scale, this is the role to do it.

Primary Responsibilities

• Define, own, and continuously evolve Fourth’s AI‑driven global QA strategy and roadmap, aligned with the Product and Technology roadmaps, to shape the strategic testing needs of the business.
• Establish the QA tools, frameworks, standards, and ways of working across our global engineering organisation to achieve our technology goals.
• Drive the unification and continuous improvement of testing processes and practices across delivery teams, partnering directly with Software Engineering Managers and Product Managers.
• Act as the principal point of escalation for complex testing and quality challenges across global Engineering teams, marshalling subject‑matter expertise to resolve them.
• Champion modern testing technologies, frameworks, and tools across Engineering, including the responsible adoption of AI.
• Define and own the quality KPIs and quality gates that measure product quality and the effectiveness of AI‑assisted delivery.
• Own the Engineering Defect Management process and the organisation‑wide approach to defect prevention.
• Lead Fourth’s QA approach for AI development end to end, covering both AI‑assisted development (such as AI code generation and developer copilots) and agentic AI development (autonomous and semi‑autonomous systems).
• Establish quality, evaluation and governance frameworks for non‑deterministic and AI‑driven systems, including test and evaluation strategies for LLM‑based and agentic features.
• Set enterprise standards for validating AI‑generated code, ensuring it meets Fourth’s quality, security, and maintainability requirements.
• Partner with Engineering, Product, and Service Delivery leadership to embed AI fluency and modern quality practice across the organisation, mentoring and uplifting QA capability globally.

Key Skills and Competencies

• Strong understanding of the testing and quality challenges of non‑deterministic, ML/LLM‑based, and agentic systems, including evaluation, guardrails, and the regression of model‑driven behaviour.
• Proven experience defining and implementing QA strategy at an organisational or enterprise scale.
• Experience establishing test automation best practice, both functional and non‑functional, across multiple teams.
• Demonstrable experience shaping or leading a QA approach for AI‑assisted and/or agentic AI development.
• Track record of defining and analysing quality KPIs and quality gates.
• Ability to influence and provide technical leadership and mentoring across senior engineering and product stakeholders.
• Experience creating and owning technical strategy and process documentation.
• Strong experience with Agile software delivery at scale.
• Excellent communication and stakeholder‑management skills, with fluent English (written and spoken).

Preferred Experience And/or Qualifications

• Familiarity with MLOps or LLMOps practices, and with monitoring and observing the behaviour of AI systems in production.
• Exposure to prompt design and prompt/response evaluation techniques for AI‑enabled features.
• Strong programming proficiency in C# or Java.
• Expertise in web and API testing, including hands‑on UI automation with modern tools such as Playwright (or comparable frameworks).
• Proven ability to design and implement robust, scalable QA test strategies and automation frameworks.
• Hands‑on experience with performance, security and non‑functional testing.
• Practical experience using AI‑assisted development tools (such as GitHub Copilot or similar), with a clear understanding of how to assure the quality, security and maintainability of AI‑generated code.
• A practical understanding of how AI is reshaping software delivery and the new quality risks it introduces.
• Experience with CI/CD tooling, e.g. Jenkins, Azure DevOps and Octopus Deploy.
• Experience with observability tooling such as Prometheus, Grafana and Sumo Logic.

Benefits

• 25 basic holidays with the option to grow up to 30 with service, birthday off and bank holidays.
• Flexible working hours and hybrid working environments.
• Wellness activities and gym subscription discounts.
• Company laptop and equipment.
• Healthcare expense claim tools.
• Support for certifications and professional growth.
• Annual meet‑ups.
• Enhanced parenting scheme.
• Cycle to work scheme and season ticket loan provided.
• Pension and life insurance options.
• On‑demand pay tools to access salary before the end of the month.

All personal data provided by you in your application will be processed by Fourth LTD solely for our recruitment purposes. More information on Fourth LTD's privacy policy for job candidates is available here and by applying you confirm that you have read it.

Fourth is an Equal Opportunity Employer

All qualified applicants will receive consideration without discrimination because of sex, gender identity, gender expression, sexual orientation, marital status, race, colour, age, national origin, military status, religion, or disability or any other legally protected status.