AI Test Lead: Responsible AI & MLOps

Yorkshire Building Society; Leeds; Hybrid; GBP 70,000 - 100,000; Full time

Benefits offered by this job

Holiday 25 days + BH

On-target bonus 7% (up to 15%)

Pension contributions up to 11%

Private medical insurance

Self-service benefits portal

Job summary

Yorkshire Building Society invites an AI Test Lead to define and lead testing for AI-enabled systems within our Data & AI function. This hybrid role involves onsite presence in Bradford and Leeds with remote work options, focusing on robust QA practices and responsible AI considerations.

You will drive testing frameworks for probabilistic AI outputs, embed bias and safety tests, and integrate automation into DevOps/MLOps pipelines in a regulated financial services environment.

Qualifications

• Experience in testing and QA for complex systems.
• Knowledge of AI/ML tech, LLMs, and RAG.
• Familiarity with test automation and CI/CD pipelines.

Responsibilities

• Develop AI testing frameworks for non-deterministic outputs and variability.
• Embed responsible AI and risk testing with bias, fairness, and safety tests.
• Implement automated AI testing and integrate into DevOps/MLOps pipelines.
• Collaborate with delivery and governance teams to ensure compliance.

Skills

QA testing

AI

DevOps

MLOps

Azure

Tools

Azure AI Services

Copilot

Test automation

Job description

Yorkshire Building Society invites an AI Test Lead to define and lead testing for AI-enabled systems within our Data & AI function. This hybrid role involves onsite presence in Bradford and Leeds with remote work options, focusing on robust QA practices and responsible AI considerations.

You will drive testing frameworks for probabilistic AI outputs, embed bias and safety tests, and integrate automation into DevOps/MLOps pipelines in a regulated financial services environment.