AI Quality Engineering Lead

Phase2

Town of Montana (WI)

Hybrid

USD 150,000 - 190,000

Full time

14 days+

Application generator

A complete application in a minute — tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Phase2 is seeking a Lead AI Quality Engineer to steer software quality in an AI-driven environment. You will own quality strategy, define standards, and lead a cross-functional team, while collaborating with clients to elevate QA practices.

The role blends leadership, hands-on testing, and strategic guidance in a hybrid, remote-first setting. You will mentor the QA team, advance AI testing methodologies, and ensure responsible use of AI tools.

Qualifications

8+ years of software QA experience, ideally with AI/ML features.
Experience managing, mentoring and coaching direct reports.
Experience improving QA methods and tools.
Proficiency in automated testing in at least one framework.
Knowledge of AI testing concepts, including bias and hallucination testing.
Strong communication with technical and non-technical stakeholders.

Responsibilities

Set the vision and roadmap for software quality across AI-enabled delivery.
Define testing standards and defaults with Engineering leadership.
Innovate approaches to accelerate QA testing and tool adoption.
Lead the use of agentic AI to write tests.
Advise on QA involvement across the SDLC and projects.
Socialize QA approach across the company and client teams.
Oversee agentic QA tooling and maintain internal QA frameworks.
Test Phase2-built AI tools and evaluate performance and bias.
Develop guidelines for testing AI features (LLMs, agentic workflows, ML components).
Keep AI testing knowledge current and share insights with the team.