AI Test Lead

Yorkshire Building Society; Leeds; Hybrid; GBP 65.000 - 90.000; Vollzeit

Benefits dieser Stelle

Holiday entitlement

Annual bonus

Pension plan

Private medical insurance

Benefits portal

Zusammenfassung

Yorkshire Building Society is seeking an AI Test Lead to define and lead testing and quality assurance for AI-enabled systems. This hybrid role involves working from home with onsite presence in Bradford and Leeds offices.

The role focuses on testing AI with probabilistic behaviour, bias and risk testing, and integration into DevOps/MLOps pipelines within a regulated financial services environment.

Qualifikationen

• Strong experience in testing and quality assurance for complex systems.
• Working knowledge of AI and ML technologies, including large language models and Retrieval-Augmented Generation (RAG).
• Familiarity with test automation, DevOps pipelines, and MLOps practices.
• Experience working with Microsoft Azure, Azure AI services, and Copilot-based solutions.
• Strong collaboration and communication skills, including engagement with non-technical stakeholders.

Aufgaben

• Develop AI testing frameworks and methodologies that account for non-deterministic behaviour, probabilistic outputs, and variability in AI responses.
• Embed responsible AI and risk testing by defining test cases that assess bias, fairness, transparency, safety, and the effectiveness of AI guardrails.
• Implement automated AI testing capabilities by developing test automation and integrating testing activities into DevOps and MLOps pipelines.
• Collaborate across delivery and governance teams to ensure testing practices align with engineering standards, regulatory expectations, and enterprise assurance requirements.

Kenntnisse

QA for complex systems

AI/ML knowledge

Test automation

DevOps practices

MLOps

Collaboration with stakeholders

Tools

Azure

Copilot-based solutions

Jobbeschreibung

Are you passionate about making a difference, using your skills & experience to be a key enabler of AI-driven transformation? If so, we may have the role for you!

We are excited to share an incredible opportunity for an AI Test Lead to join our team in Data & AI. As our newest AI Test Lead, you will be responsible for defining and leading testing and quality assurance practices for AI-enabled systems and solutions.

This is a hybrid role with working from home plus some onsite presence in our Bradford & central Leeds offices.

About the role

The role addresses the unique challenges of testing AI systems, including probabilistic behaviour, variability in outputs, and responsible AI considerations. Through robust testing frameworks and collaboration with delivery and governance teams, the role supports compliant and trustworthy deployment of AI solutions within a regulated financial services environment.

You will-

• Develop AI testing frameworks and methodologies that account for non‑deterministic behaviour, probabilistic outputs, and variability in AI responses.
• Embed responsible AI and risk testing by defining test cases that assess bias, fairness, transparency, safety, and the effectiveness of AI guardrails.
• Implement automated AI testing capabilities by developing test automation and integrating testing activities into DevOps and MLOps pipelines.
• Collaborate across delivery and governance teams to ensure testing practices align with engineering standards, regulatory expectations, and enterprise assurance requirements.

About You

You will have –

• Strong experience in testing and quality assurance for complex systems
• Working knowledge of AI and machine learning technologies, including large language models and Retrieval‑Augmented Generation (RAG)
• Familiarity with test automation, DevOps pipelines, and MLOps practices
• Experience working with Microsoft Azure, Azure AI services, and Copilot‑based solutions
• Strong collaboration and communication skills, including engagement with non‑technical stakeholders

About us

At YBS we don’t have shareholders, we have members. We care about people and that includes you. So here, you’ll be respected for who you are, you’ll be able to bring your whole self to work, and you’ll have everything you need to build a long and rewarding career.

Flexibility.

We offer a range of flexible working options without unsocial hours, which can help you find a healthy work-life balance. Whether it’s finishing early for an appointment or fitting your studies and hobbies in around your working hours, we can help make it work for you. We’re happy to talk flexible working and you can reach out to us anytime to discuss this during the recruitment process.

Development.

We want you to feel challenged here. Whatever your ambitions, we’re committed to helping you develop your skills and move ahead in your career journey. From day one, we’ll set you up for success with a brilliant induction and full training plan, all designed to get you up to speed quickly.

Inclusivity.

We’re passionate about creating an inclusive environment where everyone is welcome and able to be themselves. If you’ll go the extra mile to help our members, you’ll fit right in and feel you belong here.

About our Benefits

We offer lots of fantastic rewards that you can make the most of in and out of work, including:

• Holiday. You’ll get 28 days plus Bank Holidays, as well as the option to buy up to a further 5 days.
• Bonus. At YBS we work collaboratively and share in our success together, so when we reach our goals were all rewarded with an on target bonus of 7% of eligible pay (with the opportunity to earn up to a maximum 15%).
• Pension. We know how important it is to save towards the future, that’s why we’ll contribute up 11% into your YBS pension.
• Healthcare. Health and wellbeing are an important part of life at YBS, when you join us you’ll have access to a range of health benefits to suit your life including private medical insurance, dental and healthcare plans.
• My Benefits. When you join YBS you’ll have access to our self-service benefits portal, where you can access a range of retail, hospitality and health discounts.

If you have any questions about this role, please contact [e-mail]