GenAI Test Lead

Redefined Ltd; London; On-site; GBP 60,000 - 90,000; Full time

Job summary

A leading company seeks a Gen AI Test Lead in London to shape the testing strategy for GPT applications. You will be responsible for developing test plans and leading QA lifecycle while ensuring compliance with AI governance. Ideal candidates will possess strong skills in test automation and a solid understanding of AI ethics.

Qualifications

• Proven experience as a Test Lead in GenAI or LLM-based applications.
• Strong understanding of large language models.
• Skilled in test automation and AI governance.

Responsibilities

• Define testing strategy for GPT-based applications.
• Lead end-to-end QA lifecycle for GenAI features.
• Integrate testing into CI/CD pipelines.

Skills

Test Lead experience

LLM understanding

Prompt engineering

Test automation

AI ethics knowledge

Analytical skills

Communication skills

Tools

Python

PyTest

Postman

LangChain testing utilities

Job description

Job Title

Gen AI Test Lead

Get The Future You Want

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and build a more sustainable, more inclusive world.

Your Role

• Define and document the testing strategy for GPT-based applications, covering functional, non-functional, and ethical dimensions.
• Develop test plans, cases, and data focused on LLM-specific behaviors like prompt engineering, hallucination detection, and response consistency.
• Lead the end-to-end QA lifecycle for GenAI features, from requirements to post-deployment monitoring.
• Establish quality gates and acceptance criteria for LLM outputs, ensuring accuracy, safety, and fairness.
• Identify and implement AI-specific testing tools (e.g., prompt testing, adversarial testing).
• Integrate testing into CI/CD pipelines in collaboration with MLOps and DevOps teams.
• Ensure compliance with AI governance, data privacy, and responsible AI standards.
• Validate API integrations, including authentication, rate limits, and error handling.

Your Profile

• Proven experience as a Test Lead, preferably in GenAI or LLM-based applications.
• Strong understanding of LLMs, prompt engineering, and generative AI workflows.
• Skilled in test automation using tools like Python, PyTest, Postman, and LangChain testing utilities.
• Familiarity with OpenAI APIs, model fine-tuning, and evaluation techniques.
• Knowledge of AI ethics, bias detection, and responsible AI practices.
• Experience in risk identification and mitigation for hallucinations, bias, and toxicity in LLMs.
• Excellent analytical and communication skills for cross-functional collaboration.
• Ability to provide clear reporting on test coverage, quality metrics, and model behavior insights.

About Capgemini

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than fifty countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fuelled by its market-leading capabilities in AI, cloud, and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of €22.5 billion.

Get The Future You Want | www.capgemini.com