Remote | QA Engineer — $90–$175/hour

engineeringjobs.net, Inc.; United States; Remote; USD 124,000 - 241,000; Part time

Job summary

24-MAG LLC is offering a specialised part-time consulting opportunity for experienced QA professionals to contribute to an AI training and evaluation project. You will evaluate AI outputs, design test scenarios, review defects, and provide detailed written feedback to improve software and AI quality.

Ideal candidates have hands-on QA experience, knowledge of test-case design and testing strategies, and proficiency with modern QA tools.

Qualifications

• Proven professional experience as a QA Engineer, SDET, Test Engineer, QA Analyst, or in a closely related software quality role.
• Strong understanding of QA fundamentals including test-case design, regression testing, bug tracking, and manual and automated testing strategies.
• Hands-on experience with testing or automation tools such as Selenium, Playwright, Cypress, Appium, Postman, Jira, TestRail, Zephyr, BrowserStack, or comparable platforms.

Responsibilities

• Evaluate and rate AI-generated technical outputs against defined quality criteria and evaluation rubrics.
• Identify responses that appear correct at first glance but contain inaccuracies, omissions, or incomplete reasoning.
• Assess outputs for technical correctness, completeness, clarity, and adherence to requirements.
• Apply consistent judgement across high-volume evaluation workflows.
• Document quality concerns and recurring failure patterns clearly.

Skills

QA fundamentals

Test-case design

Manual testing

Automation testing

Bug tracking

English proficiency

Attention to detail

Tools

Selenium

Playwright

Cypress

Appium

Postman

Jira

TestRail

Zephyr

BrowserStack

Job description

We are sharing a specialised part-time consulting opportunity for experienced QA professionals with strong expertise in software testing, quality assurance, test-case design, bug analysis, and manual or automated testing to contribute to an advanced AI training and technical evaluation project.

Selected professionals will evaluate AI-generated technical outputs, design and assess comprehensive test scenarios, review defects and documentation, and provide detailed written feedback that supports improvements in software and AI system quality. The work combines traditional QA practices with structured evaluation and rubric-based assessment. No prior experience in AI is required.

Key Responsibilities

AI Output & Quality Evaluation

• Evaluate and rate AI-generated technical outputs against defined quality criteria and evaluation rubrics
• Identify responses that appear correct at first glance but contain inaccuracies, omissions, or incomplete reasoning
• Assess outputs for technical correctness, completeness, clarity, and adherence to requirements
• Apply consistent judgement across high-volume evaluation workflows
• Document quality concerns and recurring failure patterns clearly

Test Case Design & Software Validation

• Design and assess comprehensive test cases covering functional, regression, negative, boundary, and edge-case scenarios
• Review software behaviour against expected requirements and acceptance criteria
• Apply systematic testing approaches to identify defects and reliability issues
• Evaluate both common and less obvious failure conditions
• Maintain strong coverage and consistency across manual and automated testing workflows

Bug Analysis & Defect Documentation

• Review bug reports and test documentation for reproducibility, completeness, and appropriate severity
• Identify, isolate, and document software defects with precise reproduction steps
• Ensure issue descriptions contain sufficient technical detail for efficient investigation
• Use structured bug-tracking and test-management processes to record findings
• Assess whether reported defects accurately reflect system behaviour and impact

Automation, Feedback & QA Collaboration

• Work with automation frameworks and testing tools to support efficient quality evaluation
• Provide detailed, actionable written feedback and annotations that developers can use without additional clarification
• Collaborate with project teams to refine testing standards, rubrics, and evaluation guidelines
• Help improve QA methodologies and consistency across project workflows
• Contribute to continuous improvement of testing processes and technical quality standards

Ideal Profile

• Proven professional experience as a QA Engineer, SDET, Test Engineer, QA Analyst, or in a closely related software quality role
• Strong understanding of QA fundamentals including test-case design, regression testing, bug tracking, and manual and automated testing strategies
• Hands-on experience with testing or automation tools such as Selenium, Playwright, Cypress, Appium, Postman, Jira, TestRail, Zephyr, BrowserStack, or comparable platforms
• Strong analytical and problem-solving skills with meticulous attention to detail
• Ability to identify subtle defects, ambiguous behaviour, incomplete requirements, and inconsistent system outputs
• Strong written English communication skills at B2 level or above
• Ability to provide precise, actionable technical feedback
• Prior paid experience with human data annotation, RLHF, AI response evaluation, model evaluation, or rubric-based grading is advantageous but not required
• Practical, demonstrable QA experience is prioritised over formal academic credentials
• Reliable internet connection and readiness to begin promptly
• No prior experience in AI research or model training is required

Engagement Details

• Part-time independent contractor engagement
• Fully remote
• Compensation: $90--$175/hour
• Work will involve AI-output evaluation, test-case design, manual and automated testing, bug analysis, defect documentation, and structured QA feedback
• Strong professional software-testing experience is central to this engagement
• Prior human-data or AI-evaluation experience is beneficial but not mandatory
• No formal degree is required
• Project scope, workload, timing, and evaluation standards may evolve depending on project requirements
• Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party

About the Platform

This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.