Remote AI Agent Evaluation Specialist

Mindrift; Johannesburg; Remote; ZAR 238,959 - 477,918; Part time

Benefits offered by this job

Flexible remote work

Competitive hourly rates

Experience in advanced AI projects

Job summary

A tech consulting firm is seeking an AI Agent Evaluation Analyst for a part-time, flexible role. This opportunity is ideal for individuals with strong analytical and detail-oriented skills who enjoy evaluating complex systems and contributing to advanced AI projects. Candidates will collaborate with developers and QA teams to improve AI agents. A background in consulting, logic puzzles, or policy evaluation is advantageous. Competitive rates may go up to $20/hour.

Qualifications

• Excellent analytical thinking about complex systems.
• Strong attention to detail in spotting contradictions.
• Familiarity with structured data formats like JSON/YAML.
• Ability to assess scenarios holistically.

Responsibilities

• Review evaluation tasks for logic and completeness.
• Identify inconsistencies and unclear decision points.
• Define clear expected behaviors for AI agents.
• Annotate cause-effect relationships and reasoning paths.
• Think through complex systems to ensure proper testing.
• Collaborate with QA, writers, or developers for refinements.

Skills

Analytical thinking

Attention to detail

Communication

Structured data formats understanding

Education

Relevant degree or experience

Job description

A tech consulting firm is seeking an AI Agent Evaluation Analyst for a part-time, flexible role. This opportunity is ideal for individuals with strong analytical and detail-oriented skills who enjoy evaluating complex systems and contributing to advanced AI projects. Candidates will collaborate with developers and QA teams to improve AI agents. A background in consulting, logic puzzles, or policy evaluation is advantageous. Competitive rates may go up to $20/hour.