Senior GenAI QA Engineer: AI Test Automation Lead

Anaplan; Greater London; On-site; GBP 70,000 - 120,000; Full time

Job summary

Anaplan is seeking a Senior SDET for AI to lead QA for GenAI-powered features. You will design test plans, build automated Python tests, and set evaluation metrics ensuring accuracy, safety, and reliability of AI components.

Part of a collaborative team focusing on scalable testing strategies, advocating for robust processes, and maintaining high-quality datasets across production workloads.

Qualifications

• Experience with GenAI technologies including LLMs and prompt engineering.
• Strong Python coding and test automation background.
• Proven ability to design test cases for non-deterministic systems.

Responsibilities

• Design and architect test plans and QA metrics for LLM-powered AI features.
• Develop end-to-end automated tests in Python for evaluations, API/UI checks and performance.
• Create prompt testing systems and regression suites for model behavior.

Skills

GenAI expertise

Python coding

Test automation

QA methodologies

Non-deterministic testing

CI/CD

API testing

Debugging & analysis

Tools

Selenium

Pytest

Postman

REST Assured

Job description

Anaplan is seeking a Senior SDET for AI to lead QA for GenAI-powered features. You will design test plans, build automated Python tests, and set evaluation metrics ensuring accuracy, safety, and reliability of AI components.

Part of a collaborative team focusing on scalable testing strategies, advocating for robust processes, and maintaining high-quality datasets across production workloads.