Senior AI QA & Test Engineer (Remote)

TestPros; United States; Remote; USD 69,000 - 110,000; Part time

Benefits offered by this job

Remote-friendly

Occasional travel to Sterling, VA

Job summary

TestPros is seeking a Senior Artificial Intelligence System Test Engineer to evaluate AI software against intended functions, focusing on model robustness, data integrity, security and reliability. Lead test planning, execution and defect remediation for enterprise-scale programs.

Role is remote with occasional travel to TestPros labs in Sterling, VA and to customer sites. Requires 10+ years in software testing and hands-on GenAI/AI testing, CI/CD integration, and familiarity with testing guides.

Qualifications

• Bachelor's degree in computer science, engineering or a related field.
• 10+ years of Software/System Testing and QE experience.
• 4+ years defining testing strategy for large scale programs and overseeing testing phases.
• Strong understanding of testing planning and delivery challenges.
• Experience with white-box, black-box, load, coverage and regression testing.
• Proven QE transformation leadership.
• Hands-on expertise in GenAI / AI frameworks.
• Experience with cloud testing platforms (AWS/Azure/GCP).
• Experience testing AI LLM-based features (prompt validation, hallucination testing).
• Experience with CI/CD and DevOps integration (GitHub).
• Familiarity with AI-assisted defect triage tools.

Responsibilities

• Develop and execute test plans; evaluate software against the OWASP AI Testing Guide and other guidance.
• Create defect and remediation records with clear reproduction steps and severity.
• Capture test evidence and document defects; track status and root cause.
• Report results to senior management and stakeholders.

Skills

10+ years testing QE

Testing strategy for large programs

White-box / Black-box testing

Python / NLP (NLTK)

AI/GenAI testing frameworks

Cloud testing (AWS/Azure/GCP)

LLM-based testing & prompt validation

CI/CD / GitHub integration

Selenium / Playwright / Cypress

OWASP AI Testing Guide knowledge

LangChain / AutoGen / CrewAI / Cursor

Sauce Labs / BrowserStack

Education

Bachelor's degree in computer science, engineering or related field

Tools

LangChain

AutoGen

CrewAI

Cursor

Sauce Labs

BrowserStack

Job description

TestPros is seeking a Senior Artificial Intelligence System Test Engineer to evaluate AI software against intended functions, focusing on model robustness, data integrity, security and reliability. Lead test planning, execution and defect remediation for enterprise-scale programs.

Role is remote with occasional travel to TestPros labs in Sterling, VA and to customer sites. Requires 10+ years in software testing and hands-on GenAI/AI testing, CI/CD integration, and familiarity with testing guides.