Remote AI QA Engineer: Data & Model Evaluation

RemoteJobsOne; Los Angeles (CA); Remote; USD 87,000 - 165,000; Full time

Job summary

RemoteJobsOne is seeking QA Engineers for a fully remote contractor role focused on AI training data quality and model evaluation. You will design and run tests, annotate data, and report on AI outputs to improve performance, with an emphasis on data integrity and thorough documentation.

The position welcomes candidates worldwide, with emphasis on strong QA practices and attention to detail. Prior AI-specific experience is beneficial but not required.

Qualifications

• Hands-on software QA experience with test planning and defect reporting.
• Experience with data labeling, annotation, RLHF, or model evaluation is preferred.
• Proficiency in English at B2 level or above for collaboration and reporting.

Responsibilities

• Design, execute, and document test cases to validate AI outputs against requirements.
• Review, annotate, and label data sets to aid model training accuracy.
• Evaluate, analyze, and report on model predictions to identify errors and patterns for improvement.
• Collaborate with cross-functional teams to communicate findings and clarify requirements.
• Contribute to testing methodologies for AI-driven systems.
• Maintain detailed records of evaluations, test results, and feedback in documentation.
• Provide actionable recommendations to optimize data quality and model performance.

Skills

Quality assurance

Data labeling

Model evaluation

Test planning

Defect reporting

English B2+

Job description

RemoteJobsOne is seeking QA Engineers for a fully remote contractor role focused on AI training data quality and model evaluation. You will design and run tests, annotate data, and report on AI outputs to improve performance, with an emphasis on data integrity and thorough documentation.

The position welcomes candidates worldwide, with emphasis on strong QA practices and attention to detail. Prior AI-specific experience is beneficial but not required.