Remote QA Engineer for AI Evaluation & Testing

engineeringjobs.net, Inc.

United States

Remote

USD 124,000 - 241,000

Part time

14 days+

Application generator

A complete application in a minute — tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

24-MAG LLC is offering a specialised part-time consulting opportunity for experienced QA professionals to contribute to an AI training and evaluation project. You will evaluate AI outputs, design test scenarios, review defects, and provide detailed written feedback to improve software and AI quality.

Ideal candidates have hands-on QA experience, knowledge of test-case design and testing strategies, and proficiency with modern QA tools.

Qualifications

Proven professional experience as a QA Engineer, SDET, Test Engineer, QA Analyst, or in a closely related software quality role.
Strong understanding of QA fundamentals including test-case design, regression testing, bug tracking, and manual and automated testing strategies.
Hands-on experience with testing or automation tools such as Selenium, Playwright, Cypress, Appium, Postman, Jira, TestRail, Zephyr, BrowserStack, or comparable platforms.

Responsibilities

Evaluate and rate AI-generated technical outputs against defined quality criteria and evaluation rubrics.
Identify responses that appear correct at first glance but contain inaccuracies, omissions, or incomplete reasoning.
Assess outputs for technical correctness, completeness, clarity, and adherence to requirements.
Apply consistent judgement across high-volume evaluation workflows.
Document quality concerns and recurring failure patterns clearly.