Remote AI Evaluation Specialist (Part-Time)

Mindrift; Remote; ZAR 200 000 - 300 000; Part time

Job summary

A technology consulting firm based in Pretoria is looking for contributors to work on a project focused on quality assurance for autonomous AI agents. The role requires strong analytical skills, attention to detail, and the ability to evaluate complex systems. The project is project-based, flexible, and well-suited for students or part-time applicants. This is a remote opportunity with competitive compensation that can reach up to $22/hour based on skills and experience.

Benefits

Flexible work hours

Competitive pay up to $22/hour

Experience on advanced AI projects

Qualifications

• Excellent analytical thinking: Can reason about complex systems, scenarios, and logical implications.
• Strong attention to detail: Can spot contradictions, ambiguities, and vague requirements.
• Good communication and clear writing (in English) to document your findings.

Responsibilities

• Review evaluation tasks and scenarios for logic, completeness, and realism.
• Identifying inconsistencies, missing assumptions, or unclear decision points.
• Helping define clear expected behaviors (gold standards) for AI agents.
• Annotating cause-effect relationships, reasoning paths, and plausible alternatives.
• Working closely with QA, writers, or developers to suggest refinements or edge-case coverage.

Skills

Analytical thinking

Attention to detail

Communication (English)

Job description

A technology consulting firm based in Pretoria is looking for contributors to work on a project focused on quality assurance for autonomous AI agents. The role requires strong analytical skills, attention to detail, and the ability to evaluate complex systems. The project is project-based, flexible, and well-suited for students or part-time applicants. This is a remote opportunity with competitive compensation that can reach up to $22/hour based on skills and experience.