QA Engineer AI Training and Evaluation

OpenTrain AI; Northern (KY); Hybrid; USD 124,000 - 241,000; Full time

Job summary

OpenTrain AI is seeking about 65 QA Engineers for expert contract work on AI training and evaluation projects. Roles involve manual and automated testing, with emphasis on human data tasks such as RLHF and evaluation rating. The project spans 3–6 months with hourly rates between $90 and $175.

Applicants should have hands-on QA experience, clear communication of defects, and English proficiency. This contractor opportunity offers flexible engagement and alignment with ongoing AI research efforts.

Qualifications

• Professional experience as a QA Engineer, Software Tester, Test Engineer, or similar role.
• Prior human-data experience is mandatory for this opportunity.
• Ability to clearly explain testing decisions, defects, and expected behavior.
• English proficiency at B2 level or higher is required.

Responsibilities

• Design and execute high-quality test cases and scenarios.
• Identify, reproduce, document, and analyze defects.
• Perform functional, regression, and other relevant testing activities.
• Review AI-generated testing approaches and outputs for accuracy.
• Evaluate whether proposed solutions address bugs and requirements.
• Provide clear, structured feedback based on QA expertise.
• Contribute to QA-focused datasets and evaluation tasks for AI systems.
• Complete evaluation rating, question answering, RLHF, and related human-data tasks.

Skills

QA testing

Manual testing

Automated testing

Test design

Defect tracking

Regression testing

Analytical skills

English proficiency

Job description

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. OpenTrain AI is hiring and contracting experienced professionals for projects that help develop and evaluate next-generation artificial intelligence systems.

• Create a free OpenTrain account and apply in minutes.
• Build a profile that showcases your professional expertise and AI training experience.
• Join an expanding expert network supporting advanced AI evaluation work.

About AI Training and Evaluation

AI training is the human side of building artificial intelligence. Experts review model outputs, test proposed solutions, and provide structured feedback that helps AI systems become more accurate, reliable, and useful.

In this role, your practical software quality assurance knowledge will support evaluation rating, question answering, RLHF, and other human-data tasks. This work gives experienced QA professionals a direct way to contribute to how modern AI systems are built.

• Apply real-world QA judgment to AI-generated testing approaches and outputs.
• Help create realistic datasets and evaluation tasks for AI systems.
• Contribute to a fast-growing field alongside your existing professional work.

The QA Engineer Contract Opportunity

OpenTrain AI is seeking approximately 65 experienced QA Engineers to contribute to AI training and evaluation projects. The opportunity is intended for professionals with hands-on experience in manual or automated testing and prior experience with human data or AI-related projects.

The expected project duration is 3–6 months. Compensation is listed at $90–$175 per hour, with rates varying according to QA specialization, experience, and project requirements.

• Work type: Expert contract opportunity
• Experience level: Intermediate
• Data formats and tasks: Text, evaluation rating, question answering, and RLHF
• Contract arrangement: Contractor
• Expected project duration: 3 to 6 months

What You'll Do

You will use your QA expertise to assess testing quality, analyze technical outputs, and provide clear feedback for AI training and evaluation workflows. Your work should reflect practical software testing standards and realistic expected behavior.

• Design and execute high-quality test cases and test scenarios.
• Identify, reproduce, document, and analyze software defects.
• Perform functional, regression, and other relevant testing activities.
• Review AI-generated testing approaches and outputs for technical accuracy.
• Evaluate whether proposed solutions correctly address bugs and requirements.
• Provide clear, structured feedback based on your QA expertise.
• Contribute to realistic QA-focused datasets and evaluation tasks for AI systems.
• Complete evaluation rating, question answering, RLHF, and related human-data tasks.

Requirements

Applicants must have professional experience as a QA Engineer, Software Tester, Test Engineer, or in a similar role. Prior human data experience is mandatory, and candidates without this experience will not be considered.

You must be able to clearly explain testing decisions, defects, expected behavior, and the reasoning behind your evaluations. English proficiency must be at least B2 level, with fluent English specified for the opportunity.

• Professional software quality assurance or testing experience
• Experience in manual testing, automated testing, or both
• Experience designing test cases and managing or tracking defects
• Experience with regression testing and functional testing
• Strong analytical and problem-solving skills
• Prior human data experience, such as data labeling, data annotation, AI/ML evaluation, or RLHF
• Experience with human-data projects or similar AI training and evaluation work
• English proficiency at B2 level or above

Who Should Apply

This opportunity is suited to QA professionals who can combine rigorous testing practices with clear written evaluation. It is especially relevant if you have already worked on data labeling, annotation, AI/ML evaluation, RLHF, or other human-data projects.

Eligible applicants are located in the United States, United Kingdom, Finland, Netherlands, Ireland, India, Argentina, Belgium, Spain, Chile, Mexico, Australia, Austria, South Korea, Japan, Sri Lanka, Brazil, Canada, Colombia, Denmark, Egypt, France, Greece, Hungary, Iceland, Italy, Jordan, Monaco, Morocco, New Zealand, New Caledonia, the Philippines, Saudi Arabia, Eswatini, Sweden, Switzerland, Türkiye, the United Arab Emirates, or Vietnam.

• Approximately 65 expert QA Engineer positions are expected.
• Rates may vary based on your QA specialization, experience, and project requirements.
• Candidates without prior human data experience will not be considered.

Selection Process

Applicants move through a structured process designed to assess both general suitability and domain expertise before onboarding.

• Application
• AI interview
• Domain screening
• Onboarding call

Why Work in AI Training

AI training and data labeling are rapidly growing ways to work in technology. People with software, language, analytical, and specialist expertise help prepare examples and review outputs that shape how AI systems behave.

As a QA Engineer, you can apply skills you already use in professional testing while contributing to cutting-edge AI development. Many AI training projects can also offer flexible participation, making the field compatible with other work or commitments.

• Help influence the accuracy and reliability of modern AI systems.
• Use specialized QA knowledge in an emerging technology field.
• Gain experience with evaluation rating, question answering, and RLHF workflows.