QA Engineer – Remote

RemoteJobsOne; Los Angeles (CA); Remote; USD 87,000 - 165,000; Full time

Job summary

RemoteJobsOne is seeking QA Engineers for a fully remote contractor role focused on AI training data quality and model evaluation. You will design and run tests, annotate data, and report on AI outputs to improve performance, with an emphasis on data integrity and thorough documentation.

The position welcomes candidates worldwide, with emphasis on strong QA practices and attention to detail. Prior AI-specific experience is beneficial but not required.

Qualifications

• Hands-on software QA experience with test planning and defect reporting.
• Experience with data labeling, annotation, RLHF, or model evaluation is preferred.
• Proficiency in English at B2 level or above for collaboration and reporting.

Responsibilities

• Design, execute, and document test cases to validate AI outputs against requirements.
• Review, annotate, and label data sets to aid model training accuracy.
• Evaluate, analyze, and report on model predictions to identify errors and patterns for improvement.
• Collaborate with cross-functional teams to communicate findings and clarify requirements.
• Contribute to testing methodologies for AI-driven systems.
• Maintain detailed records of evaluations, test results, and feedback in documentation.
• Provide actionable recommendations to optimize data quality and model performance.

Skills

Quality assurance

Data labeling

Model evaluation

Test planning

Defect reporting

English B2+

Job description

IMPORTANT: after you apply, please check your email. We sendyou a link to complete your application — it is not considered until that laststep is done. If you do not see it, check your spam folder.

This is a fully remote position, open to candidates based in 55 countries, including United States, United Kingdom, Canada, Germany, India, France, Netherlands, Singapore.

Pay: $63–$120/hr

Role Type: Contractor

Location: Remote

We are engaging QA Engineers to contribute to an AI training project with a forward-thinking customer. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters. This opportunity is ideal for those passionate about quality assurance and eager to be at the forefront of AI innovation, with a focus on data integrity, annotation, and comprehensive model evaluation.

Scope of Work

• Design, execute, and document test cases and scenarios to validate AI outputs against defined requirements.
• Review, annotate, and label data sets to enhance machine learning model training accuracy.
• Evaluate, analyze, and report on model predictions, identifying inconsistencies, errors, or patterns for continuous improvement.
• Collaborate with cross-functional teams to communicate findings, raise questions, and clarify project requirements.
• Contribute to the development and refinement of testing methodologies specific to AI-driven systems.
• Maintain detailed, organized records of evaluations, test results, and feedback through written documentation.
• Provide actionable recommendations to optimize data quality and model performance.

Preferred Qualifications

• Hands-on expertise in software quality assurance, with a strong understanding of test planning and defect reporting.
• Prior experience with data labeling, annotation, reinforcement learning from human feedback (RLHF), or model evaluation.
• Proficiency in English at B2 level or above for effective collaboration and reporting.