Graduate Testing Engineer - Reply

ZipRecruiter; London; On-site; GBP 35,000 - 45,000; Full time

Job summary

A leading company in QA and software testing is seeking a Graduate AI Test Automation Engineer. The role involves developing innovative testing frameworks for generative AI agents, requiring expertise in Python and a strong interest in AI. Join a cutting-edge project with opportunities for rapid career progression in a high-impact environment.

Qualifications

• Currently pursuing or holding a Bachelor's or Master's degree (minimum 2.1 grade).
• Familiarity with LLMs and testing tools is a plus.

Responsibilities

• Develop and maintain Python-based test automation tools.
• Define and implement test strategies for LLM behavior validation.

Skills

Python

Analytical mindset

Collaborative problem-solving

Education

Bachelor's or Master's degree in Computer Science, AI

Tools

PyTest

Playwright

Job description

Job Description

Graduate AI Test Automation Engineer

About Concept Quality Reply:

Concept Quality Reply is a QA and software testing company focused on delivering high-quality digital solutions. We provide governance and production monitoring to ensure ongoing performance and compliance after release. Through advanced test automation and AI-driven testing strategies, we help organizations accelerate development, reduce risk, and ensure reliability across the entire software lifecycle. Our goal is to transform QA into a strategic driver of innovation and efficiency.

Role Overview:

We are looking for a Graduate AI Test Automation Engineer with expertise in Python and a strong interest in Artificial Intelligence who will support the development of an innovative testing framework for generative AI agents. You will be involved in designing tools for test data creation, automating evaluation processes, and contributing to AI model quality metrics (such as accuracy, bias, and consistency). This position offers the opportunity to join a cutting-edge project shaping AI testing standards, working with experts in AI, testing, and data governance, with potential for rapid career progression in a high-impact, AI-driven environment.

Responsibilities:

1. Develop and maintain Python-based test automation tools.
2. Collaborate on framework design and integrate third-party tools.
3. Define and implement test strategies for LLM behavior validation.
4. Support continuous testing, benchmarking, and KPI tracking.

About the candidate:

• Currently pursuing or holding a Bachelor's or Master's degree in Computer Science, AI, or related fields (minimum 2.1 grade).
• Strong interest in AI-driven software testing and development.
• Proficient in Python and data libraries.
• Familiarity with LLMs and testing tools (e.g., PyTest, Playwright) is a plus.
• Experience with datasets and evaluation metrics (e.g., BLEU, ROUGE) is beneficial.
• Analytical mindset and collaborative problem-solving skills.

Reply is an Equal Opportunities Employer committed to diversity and inclusion. We provide equal employment opportunities to all applicants and prohibit discrimination and harassment based on characteristics protected by law. Please inform us of any reasonable adjustments needed during the recruitment process.