Director of AI Testing & Quality

Newcode; Dublin; Hybrid; EUR 120,000 - 160,000; Full time

Job summary

Newcode in Dublin, Ireland, seeks a Head of Testing to own the vision, strategy, and execution of product testing across AI-powered offerings. You will develop frameworks, processes, and standards to evaluate AI-enabled products and ensure reliable, high-quality solutions for customers.

The role requires expertise in software testing and AI product evaluation, with a track record of building testing functions.

Qualifications

• 5+ years of experience in software testing, quality assurance, product quality, or a related technical role.
• Proven experience testing AI-powered products, machine learning systems, or LLM-based applications.
• Strong understanding of AI evaluation methodologies and how to measure AI system performance.
• Experience designing testing frameworks and quality processes from the ground up.
• Ability to translate complex technical issues into clear insights and recommendations.
• Strong analytical and problem-solving skills with high attention to detail.
• Experience working cross-functionally with Engineering, Product, and Customer-facing teams.
• Comfortable operating in a fast-paced startup environment with changing priorities.
• Familiarity with automated testing frameworks and tooling.
• Excellent written and verbal communication skills.
• Experience building quality teams or functions within a growing organization.

Responsibilities

• Develop and execute Newcode’s overall testing strategy across AI-powered products.
• Create testing frameworks and methodologies for evaluating product functionality, usability, accuracy, reliability, and performance.
• Design and implement processes for testing AI outputs, including quality benchmarks, evaluation criteria, and success metrics.
• Build structured test plans, test cases, and documentation to support product releases.
• Identify gaps, risks, and improvement opportunities through rigorous product testing.
• Partner with Engineering team to improve product quality and development processes.
• Establish repeatable workflows for regression testing, user acceptance testing, and release validation.
• Analyze testing results and provide actionable insights to improve product performance.
• Develop best practices for AI evaluation, including assessing accuracy, consistency, hallucination risks, and edge cases.
• Help define quality standards as Newcode scales its products and customer base.
• Build and evolve the testing function, including future team growth and processes.

Skills

Software testing

AI testing

Quality assurance

Cross-functional collaboration

Regression testing

Automation testing

Tools

Automation frameworks

Job description

Newcode in Dublin, Ireland, seeks a Head of Testing to own the vision, strategy, and execution of product testing across AI-powered offerings. You will develop frameworks, processes, and standards to evaluate AI-enabled products and ensure reliable, high-quality solutions for customers.

The role requires expertise in software testing and AI product evaluation, with a track record of building testing functions.