AI QA and Model Evaluation Engineer

OpenTrain AI, Inc.; United States; Remote; USD 87,000 - 165,000; Part time

Job summary

OpenTrain AI, Inc. seeks an entry-level contractor to review AI model outputs, datasets and evaluation records, applying project guidelines with clear evidence.

This remote, part-time role offers flexible hours over 1 to 3 months, paying $63–$120 per hour, with estimated earnings of $10,080–$19,200 depending on hours worked. Open to candidates in the United States, Canada, the UK, India, Japan and Brazil. English proficiency must be at B2 or above, with QA/data labeling experience helpful.

Qualifications

• Entry-level contractor role; prior AI training experience not required.
• Strong practical knowledge of software QA, especially test planning and defect reporting.
• English proficiency at B2 level or above.

Responsibilities

• Evaluate model outputs using project rubrics.
• Review source materials and model responses against project guidelines.
• Document evidence-based judgments and explain your reasoning clearly.
• Provide feedback that can help improve model behavior.
• Maintain consistent quality across repeated review tasks.

Skills

Software QA

Test planning

Defect reporting

Data labeling

Annotation

Model evaluation

RLHF

Job description

The Work

You will review AI model outputs, datasets, and evaluation records to help improve how AI systems behave. The work involves applying project guidelines consistently and explaining each judgment with clear evidence.

• Evaluate model outputs using project rubrics.
• Review source materials and model responses against project guidelines.
• Document evidence-based judgments and explain your reasoning clearly.
• Provide feedback that can help improve model behavior.
• Maintain consistent quality across repeated review tasks.

What It Pays and Takes

This is an entry-level contractor role, and prior AI training experience is not required. Strong practical knowledge of software quality assurance is the main qualification, especially test planning and defect reporting.

• Pay: $63 to $120 per hour.
• Work type: Remote, part-time contract work.
• Schedule: Flexible weekly hours over 1 to 3 months.
• Estimated project earnings: $10,080 to $19,200 based on an OpenTrain assumption of 20 hours per week over the stated project duration.
• Location: Open to candidates in the countries specified for this role, including the United States, Canada, the United Kingdom, India, Japan, and Brazil.
• Language: English proficiency at B2 level or above.
• Helpful experience: Software quality assurance, test planning, defect reporting, data labeling, annotation, RLHF, or model evaluation.

About AI Training Work

AI training work is the human review behind modern AI systems. Contributors assess model responses, label data, and give structured feedback so systems can learn from accurate, consistent examples; people with strong subject knowledge are paid for applying that judgment.