GRADUATE AI TEST AUTOMATION ENGINEER

Reply, Inc.; London; On-site; GBP 35,000 - 50,000; Full time

Job summary

A leading company in QA and software testing is seeking a Graduate AI Test Automation Engineer. This role involves developing innovative testing frameworks for generative AI agents, focusing on automation and quality metrics. Ideal candidates will have a strong background in Python and a keen interest in AI-driven testing. This position offers rapid career progression in a dynamic environment.

Qualifications

• On track to complete or have completed a degree with minimum 2.1 grade.
• Initial experience with datasets and evaluation metrics is beneficial.

Responsibilities

• Develop and maintain Python-based test automation tools.
• Define and implement test strategies for LLM behavior validation.
• Collaborate on framework design and third-party tool integration.

Skills

Python

AI-driven software testing

Analytical mindset

Education

Bachelor’s or Master’s degree in Computer Science, AI, or related field

Tools

PyTest

Playwright

Job description

Requisition ID: 10680

Posted

Years of Experience: 1

Technology

Location: (1)

Job

Concept Quality Reply is a QA and software testing company focused on delivering high-quality digital solutions. We provide governance and production monitoring to ensure ongoing performance and compliance after release. Through advanced test automation and AI-driven testing strategies, we help organizations accelerate development, reduce risk, and ensure reliability across the entire software lifecycle. Our goal is to transform QA into a strategic driver of innovation and efficiency.

Role Overview:

We are looking for a Graduate AI Test Automation Engineer with expertise in Python and a strong interest in Artificial Intelligence. You will support the development of an innovative testing framework for generative AI agents, designing tools for test data generation, automating evaluation processes, and contributing to AI model quality metrics such as accuracy, bias, and consistency. This position offers the opportunity to work on a cutting-edge project shaping AI testing standards, collaborating with experts in AI, testing, and data governance, with potential for rapid career progression in a high-impact, AI-driven environment.

Responsibilities:

• Develop and maintain Python-based test automation tools
• Collaborate on framework design and third-party tool integration
• Define and implement test strategies for LLM behavior validation
• Support continuous testing, benchmarking, and KPI tracking

About the Candidate:

• On track to complete or have already completed a Bachelor’s or Master’s degree in Computer Science, AI, or a related field (minimum 2.1 grade)
• Strong interest in AI-driven software testing and development
• Proficient in Python and data libraries
• Familiarity with LLMs and testing tools (e.g., PyTest, Playwright) is a major asset
• Initial experience with datasets and evaluation metrics (e.g., BLEU, ROUGE) is beneficial
• Analytical mindset and collaborative problem-solving approach

Reply is an Equal Opportunities Employer committed to diversity and non-discrimination in the workplace. We provide equal employment opportunities to all employees and applicants, regardless of age, sexual orientation, gender, identity, pregnancy, religion, nationality, ethnicity, disability, medical history, skin colour, marital or parental status, or any other characteristic protected by law. We are dedicated to fair recruitment practices and encourage applicants to inform us of any reasonable adjustments needed during the process.