AI Validation Engineer

Agility Partners; Cincinnati (OH); On-site; USD 95,000 - 130,000; Full time

Job summary

Agility Partners is seeking an AI Validation Engineer (SDET) to ensure quality, reliability, and performance of an API and Agentic Commerce platform. You will blend QA automation with AI validation practices, collaborating with AI Engineers and Developers to test intelligent systems and LLM-powered applications.

The role focuses on building automated testing for APIs and backend services, validating AI-driven outputs, and improving overall quality standards within a fast-paced engineering team.

Qualifications

• 5+ years of QA Automation, SDET, or Quality Engineering experience.
• Strong API testing experience (REST APIs, microservices, distributed systems).
• Hands-on experience with tools like Postman, REST Assured, Playwright, PyTest, SoapUI, or Bruno.

Responsibilities

• Build and maintain automated testing frameworks for APIs and backend services
• Create functional, integration, regression, and end-to-end test automation
• Validate REST APIs, microservices, cloud services, and CI/CD pipelines
• Test AI-powered applications, agent workflows, and LLM-generated outputs
• Identify hallucinations, reasoning failures, and unexpected agent behavior
• Develop evaluation frameworks for AI responses and recommendations
• Partner with engineering teams to establish quality standards and best practices

Skills

QA Automation

SDET

Quality Engineering

API testing

AI validation

Tools

Postman

REST Assured

Playwright

PyTest

SoapUI

Bruno

Job description

Agility Partners is seeking an AI Validation Engineer (SDET) to help ensure the quality, reliability, and performance of a cutting-edge API and Agentic Commerce platform.

In this role, you’ll combine traditional QA Automation and API Testing with emerging AI validation practices, working alongside AI Engineers and Developers to test intelligent systems, agent workflows, and LLM-powered applications.

What You’ll Do

• Build and maintain automated testing frameworks for APIs and backend services
• Create functional, integration, regression, and end-to-end test automation
• Validate REST APIs, microservices, cloud services, and CI/CD pipelines
• Test AI-powered applications, agent workflows, and LLM-generated outputs
• Identify hallucinations, reasoning failures, and unexpected agent behavior
• Develop evaluation frameworks for AI responses and recommendations
• Partner with engineering teams to establish quality standards and best practices

What We’re Looking For

• 5+ years of QA Automation, SDET, or Quality Engineering experience
• Strong API testing experience (REST APIs, microservices, distributed systems)
• Hands-on experience with tools like Postman, REST Assured, Playwright, PyTest, SoapUI, or Bruno