QA Engineer | Remote - Contract

Xperteez Technology; United States; Remote; USD 34,000 - 69,000; Part time

Job summary

Xperteez Technology Pvt Ltd is seeking a QA Engineer contractor to contribute to high-volume AI data evaluation. You will design test cases, assess AI outputs, and document defects with precise steps for rapid resolutions.

This remote role emphasizes human data annotations and RLHF-related evaluation, with flexible start times and fast onboarding. Applicants should bring strong QA fundamentals, experience across manual and automated testing, and familiarity with toolchains like Selenium,

Qualifications

• Proven experience as a QA Engineer, SDET, QA Analyst, or similar role in software QA.
• Strong QA fundamentals: test design, bug tracking, regression testing, and both manual/automated testing.
• Hands-on with automation frameworks and test management tools such as Selenium, Playwright, Cypress, Appium, Postman, Jira, TestRail, Zephyr, or BrowserStack.
• Prior paid human data experience supporting AI training—annotation, labeling, RLHF, or rubric-based grading is required.
• Excellent analytical skills and clear written English communication (B2 level or above).
• No formal degree required; practical testing experience matters.

Responsibilities

• Evaluate and rate AI-generated outputs against quality criteria and rubrics.
• Design and assess test cases for functional, regression, and edge cases.
• Review bug reports and test docs for reproducibility and proper severity.
• Document defects with precise reproduction steps and tracking.
• Provide actionable written feedback to help developers resolve issues quickly.
• Collaborate to refine evaluation guidelines and improve testing standards.

Skills

Quality Assurance

Software Testing

Test Case Design

Automation Testing

Regression Testing

Bug Tracking

Test Management Tools

Selenium

Manual & Automated Testing

Bug Reporting

Quality Evaluation

Human Data

Tools

Selenium

Playwright

Cypress

Appium

Postman

Jira

TestRail

Zephyr

BrowserStack

Job description

Role Title: QA Engineer

Role Type: Contractor

Location: Remote

We are engaging QA Engineers to contribute expertise in software quality evaluation for a customer's high-volume project. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters. This opportunity is open to specialists who have both strong QA backgrounds and prior paid experience with human data annotation, RLHF, model output evaluation, or rubric-based grading for AI training. Software QA experience alone, without a human-in-the-loop AI evaluation component, does not count toward this requirement.

Required Skills

• Quality Assurance
• Software Testing
• Test Case Design
• Automation Testing
• Regression Testing
• Bug Tracking
• Test Management Tools
• Selenium
• Manual & Automated Testing
• Bug Reporting
• Quality Evaluation
• Human Data

Scope of Work

• Evaluate and rate AI-generated technical outputs against defined quality criteria and rubrics, identifying answers that may appear correct but are actually inaccurate or incomplete.

• Design and assess comprehensive test cases for functional, regression, and edge-case scenarios, including negative and boundary tests.

• Review bug reports and test documentation to ensure issues are reproducible, documentation is complete, and severity is properly assessed.

• Identify, isolate, and document defects with precise reproduction steps and utilize structured tracking mechanisms.

• Provide detailed, actionable written feedback and annotations to support efficient resolution by developers without the need for clarification.

• Collaborate with project teams to refine evaluation guidelines and ensure continuous improvement of testing standards and methodologies.

Preferred Qualifications

• Proven professional experience as a QA Engineer, SDET, Test Engineer, QA Analyst, or similar role in software quality assurance.

• Strong command of QA fundamentals, including test case design, bug tracking, regression testing, and both manual and automated testing strategies.

• Hands-on experience with automation frameworks and test management tools such as Selenium, Playwright, Cypress, Appium, Postman, Jira, TestRail, Zephyr, or BrowserStack (or similar tools).

• Prior paid human data experience supporting AI training—such as annotation, labeling, RLHF, AI response evaluation, model evaluation, or rubric-based grading is a firm project requirement.

• Excellent analytical and problem-solving skills with meticulous attention to detail.

• Ability to communicate complex findings clearly in written English, providing specific, actionable feedback (B2 level or above).

• No formal degree required—practical, demonstrable testing experience takes precedence.

• Reliable internet connection and readiness to begin promptly.