QA Engineer | Remote

Crossing Hurdles; United States; Remote; AUD 90,000 - 130,000; Full time

Job summary

Crossing Hurdles in Australia is seeking an AI QA specialist to design, execute, and document test cases that validate outputs against requirements. You will review data sets, annotate, and evaluate model predictions to identify inconsistencies and opportunities for improvement.

Collaborating with cross-functional teams, you will refine testing methodologies for AI-driven systems and maintain clear records of evaluations and feedback to boost data quality and model performance.

Qualifications

• Hands-on expertise in software quality assurance, with a strong understanding of test planning and defect reporting.
• Strong relevant experience with data labeling, annotation, reinforcement learning from human feedback (RLHF), or model evaluation.
• Proficiency in English at B2 level or above for effective collaboration and reporting.

Responsibilities

• Design, execute, and document test cases and scenarios to validate AI outputs against defined requirements.
• Review, annotate, and label data sets to enhance machine learning model training accuracy.
• Evaluate, analyze, and report on model predictions, identifying inconsistencies, errors, or patterns for continuous improvement.
• Collaborate with cross‑functional teams to communicate findings, raise questions, and clarify project requirements.
• Contribute to the development and refinement of testing methodologies specific to AI‑driven systems.
• Maintain detailed, organized records of evaluations, test results, and feedback through written documentation.
• Provide actionable recommendations to optimize data quality and model performance.

Skills

Test planning

Defect reporting

Job description

• Design, execute, and document test cases and scenarios to validate AI outputs against defined requirements.
• Review, annotate, and label data sets to enhance machine learning model training accuracy.
• Evaluate, analyze, and report on model predictions, identifying inconsistencies, errors, or patterns for continuous improvement.
• Collaborate with cross‑functional teams to communicate findings, raise questions, and clarify project requirements.
• Contribute to the development and refinement of testing methodologies specific to AI‑driven systems.
• Maintain detailed, organized records of evaluations, test results, and feedback through written documentation.
• Provide actionable recommendations to optimize data quality and model performance.

Requirements

• Hands‑on expertise in software quality assurance, with a strong understanding of test planning and defect reporting.
• Strong relevant experience with data labeling, annotation, reinforcement learning from human feedback (RLHF), or model evaluation.
• Proficiency in English at B2 level or above for effective collaboration and reporting.