QA Engineer

iFindTech Ltd; Greater London; On-site; GBP 60,000 - 90,000; Full time

Job summary

iFindTech Ltd, a fast-growing AI company in the United Kingdom, seeks a QA Engineer to own how the quality of its AI products is measured, tested and improved. You’ll build evaluation pipelines, datasets and automated tests at the intersection of AI evaluation, software engineering and product quality.

Collaborating with engineering and product teams, you will identify failure modes, derive root causes and translate findings into measurable improvements, ensuring accuracy, reliability and

Qualifications

• Must have strong software engineering or technical background with hands-on experience in AI/ML systems.
• Experience building testing, evaluation or quality frameworks for AI-powered products.
• Proficiency in Python or similar programming, capable of building evaluation tooling and automation.
• Solid understanding of LLMs and Generative AI, including testing and evaluation approaches.
• Experience creating evaluation datasets, test cases and regression suites for complex AI features.
• Ability to analyze model outputs, identify failure patterns and root causes, and translate into improvements.
• Strong focus on accuracy, reliability and consistency when evaluating AI systems.
• Excellent analytical and problem-solving skills, especially for ambiguous or hard-to-measure problems.
• Comfortable communicating technical findings with Engineering and Product teams.

Responsibilities

• Own the quality measurement framework for AI products, including data pipelines, tests and evaluation tooling.
• Collaborate with Engineering and Product teams to identify failure modes and derive actionable improvements.
• Develop datasets, test cases and regression suites to ensure AI features meet production-ready standards.
• Design and run experiments to benchmark model outputs and refine evaluation metrics.
• Communicate findings clearly to technical and non-technical stakeholders and advocate for quality improvements.

Skills

AI/ML background

AI evaluation frameworks

Python development

LLMs / Generative AI

Evaluation datasets

Model output analysis

Accuracy & reliability

Analytical problem solving

Cross-functional communication

Job description

We’re working with a fast-growing AI company looking for a QA Engineer to own how the quality of its AI products is measured, tested and improved. This isn’t traditional QA — you’ll work at the intersection of AI evaluation, software engineering and product quality, building the frameworks, datasets and automated tests that determine whether LLM-powered features are genuinely accurate, reliable and ready for production. You’ll work closely with engineering and product teams to identify where AI systems fail, understand why, and turn those findings into measurable improvements.

Must Haves

• Strong software engineering or technical background, ideally with hands-on experience working with AI/ML systems
• Experience building testing, evaluation or quality frameworks for AI-powered products
• Strong Python or similar programming experience, with the ability to build evaluation tooling and automation
• Good understanding of LLMs and Generative AI, including how model outputs should be tested and evaluated
• Experience creating evaluation datasets, test cases and regression suites for complex AI features
• Ability to analyse model outputs, identify failure patterns and root causes, and turn findings into measurable improvements
• Strong understanding of accuracy, reliability and consistency when evaluating AI systems
• Excellent analytical and problem-solving skills, particularly when dealing with ambiguous or difficult-to-measure problems
• Comfortable working closely with Engineering and Product teams and communicating technical findings clearly

Nice to Haves

• Experience evaluating RAG / retrieval systems, including the quality and relevance of retrieved information
• Experience testing AI agents, tool-calling or multi-step AI workflows
• Knowledge of different LLM evaluation methodologies, metrics and benchmarking approaches
• Experience implementing automated AI regression testing within development or CI/CD workflows
• Experience with human-in-the-loop evaluation alongside automated testing
• Knowledge of prompt evaluation and comparing changes across different models or prompts
• Experience building internal AI evaluation tooling, dashboards or monitoring
• Previous experience working with AI products where accuracy and reliability are particularly important