Senior AI Quality Engineer

Change Recruitment; City of Melbourne; On-site; AUD 120,000 - 160,000; Full time

Job summary

Change Recruitment is partnering with a client in Melbourne to hire a hands-on Senior AI Quality Engineer. You will own quality across AI-powered products, building and extending Playwright automation frameworks and evaluating AI outputs for accuracy and safety.

You will lead test strategy, conduct exploratory and API testing, define acceptance criteria, and communicate quality risks to engineers and business stakeholders.

Qualifications

• Hands-on quality engineer with Playwright automation experience.
• Experience testing AI-powered products or intelligent automation solutions.
• Strong API and integration testing skills; solid manual testing capability.

Responsibilities

• Build and extend Playwright automation frameworks from scratch.
• Test AI outputs for accuracy, grounding, hallucinations, safety, and suitability.
• Perform exploratory, functional, API, integration and E2E testing.
• Define acceptance criteria and test approaches for complex AI behaviours.
• Lead defect triage, quality discussions and release-readiness reviews.
• Present findings and communicate quality risks to technical and business stakeholders.

Skills

Playwright automation

AI testing

API testing

Exploratory testing

Quality ownership

Tools

Playwright

Job description

I’m partnering with a client looking for a hands-on Senior AI Quality Engineer to take ownership of quality across AI-powered products.

This role combines deep automation expertise with practical experience testing AI solutions. You’ll work closely with AI engineers and product teams, challenge requirements, build frameworks from scratch and help determine whether AI outputs are reliable and ready for production.

What you’ll be working on:

• Building and extending Playwright automation frameworks from the ground up.
• Testing AI outputs for accuracy, grounding, hallucinations, safety, and suitability for their intended use.
• Hands-on exploratory, functional, API, integration and end-to-end testing.
• Defining acceptance criteria and test approaches for complex AI behaviours.
• Leading defect triage, technical quality discussions and release-readiness reviews.
• Presenting findings, demonstrating solutions and communicating quality risks to technical and business stakeholders.

What I’m looking for:

• Strong hands-on Playwright expertise and experience building automation frameworks from scratch.
• Proven experience testing AI-powered products or intelligent automation solutions.
• Strong API and integration testing skills, alongside solid manual and exploratory testing capability.
• The confidence to question requirements, troubleshoot problems and influence technical decisions.
• Someone who takes ownership, learns quickly and stays actively involved in delivery.

If you enjoy working closely with AI engineers and combining technical testing with broader quality ownership, I’d be keen to have a chat.