Test Lead

Pinnacle Talent Placement; United States; Remote; USD 140,000 - 180,000; Full time

Job summary

Pinnacle Talent Placement is seeking an AI Test Lead for remote work with some travel to Atlanta, GA. You will own the quality strategy for AI-powered applications and models, defining end-to-end AI testing approaches and leading execution with modern AI tools.

The role requires 10+ years in software QA with 2+ years in AI/ML testing, hands-on testing tools, and strong leadership skills to mentor testers and collaborate across data science, engineering, and product teams.

Qualifications

• 10+ years in software QA/testing, with 2+ years hands-on testing AI/ML applications and models.
• Hands-on with AI-based testing tools and test automation.
• Ability to create and drive AI testing strategy from the ground up.

Responsibilities

• Define and own the AI testing strategy for applications and models.
• Lead AI application testing across functional, integration, and user-facing behavior.
• Perform model testing and validation (accuracy, robustness, bias, drift, hallucination, and performance).
• Leverage AI-powered testing tools to design, automate, and accelerate test coverage.
• Establish Agentic test frameworks, quality gates, and acceptance criteria for AI features.
• Collaborate with data science, engineering, and product teams to embed quality early.
• Report on quality metrics, risks, and readiness for release.

Skills

QA testing

AI/ML testing

Test automation

CI/CD

MLOps

Team leadership

Tools

LangFuse

Langsmith

TestRigor

Functionize

Job description

Status: Must be legally eligible to work where you live.

Resume must be in English

Location: Remote, some travel to Atlanta, GA

About the Role

We are looking for an AI Test Lead with 10+ years of overall testing experience, including 2+ years testing AI/ML systems, to own the quality strategy for our AI-powered applications and models. You will define how we test AI systems end-to-end, lead testing execution using modern AI tools, and set the standard for reliable, trustworthy AI delivery.

Key Responsibilities

• Define and own the AI testing strategy for applications and models.
• Lead AI application testing across functional, integration, and user-facing behavior.
• Perform model testing and validation (accuracy, robustness, bias, drift, hallucination, and performance).
• Leverage AI-powered testing tools to design, automate, and accelerate test coverage.
• Establish Agentic test frameworks, quality gates, and acceptance criteria for AI features.
• Collaborate with data science, engineering, and product teams to embed quality early.
• Report on quality metrics, risks, and readiness for release.

Required Skills & Experience

• 10+ years in software QA/testing, with 2+ years hands-on testing AI/ML applications and models.
• Hands-on with AI-based testing tools and test automation.
• Ability to create and drive AI testing strategy from the ground up.
• Strong understanding of model evaluation metrics and AI failure modes.
• Proven experience leading QA teams and mentoring testers.
• Familiarity with CI/CD and MLOps pipelines.

Nice to have:

LangFuse, Langsmith, Test Rigor of Functionize experience