Remote AI Testing & Quality Engineer

INSPYR Solutions; Merrifield (VA); Remote; USD 76,000 - 85,000; Full time

Job summary

INSPYR Solutions is seeking a hands-on AI Test Engineer to establish and scale AI testing and evaluation capabilities. This remote ET-hours role collaborates with AI engineers, platform teams, and product stakeholders to build automated testing frameworks, evaluate AI-driven solutions, and ensure quality and performance.

The ideal candidate will design automated tests, execute AI evaluations, analyze model outputs for defects and bias, and help define best practices for AI quality assurance

Qualifications

• Bachelor's degree in a related technical field.
• 3+ years of software testing, quality engineering, or AI/ML testing.
• Experience designing automated test strategies for complex software applications.
• Strong programming in Python.
• Experience with APIs, CI/CD, and test automation.

Responsibilities

• Design, develop, and maintain AI testing frameworks for generative and predictive AI.
• Create and run automated test suites validating AI model behavior and performance.
• Develop automated evaluation pipelines integrated with CI/CD workflows.
• Perform functional, regression, performance, and scenario-based testing of AI-enabled apps.
• Analyze AI outputs to identify defects, bias, and quality concerns.
• Build and maintain test datasets, prompts, and evaluation artifacts.
• Collaborate with software and AI engineers to improve model quality.
• Define, measure, and report AI quality metrics such as latency and reliability.
• Document test strategies, results, and recommendations for stakeholders.
• Support AI governance and responsible AI efforts in evaluations.

Skills

Python

Test automation

CI/CD

Data validation

Analytical skills

Problem solving

Communication

Education

Bachelor's in CS/IS/SE

Tools

Azure AI

GitHub Copilot

OpenAI

Hugging Face

LangChain

Job description

INSPYR Solutions is seeking a hands-on AI Test Engineer to establish and scale AI testing and evaluation capabilities. This remote ET-hours role collaborates with AI engineers, platform teams, and product stakeholders to build automated testing frameworks, evaluate AI-driven solutions, and ensure quality and performance.

The ideal candidate will design automated tests, execute AI evaluations, analyze model outputs for defects and bias, and help define best practices for AI quality assurance