Remote AI Agent Evaluation Specialist

Mindrift

Johannesburg

Remote

ZAR 238,959 - 477,918

Part time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Flexible remote work
Competitive hourly rates
Experience in advanced AI projects

Job summary

A tech consulting firm is seeking an AI Agent Evaluation Analyst for a part-time, flexible role. This opportunity is ideal for individuals with strong analytical and detail-oriented skills who enjoy evaluating complex systems and contributing to advanced AI projects. Candidates will collaborate with developers and QA teams to improve AI agents. A background in consulting, logic puzzles, or policy evaluation is advantageous. Competitive rates may go up to $20/hour.

Qualifications

  • Excellent analytical thinking about complex systems.
  • Strong attention to detail in spotting contradictions.
  • Familiarity with structured data formats like JSON/YAML.
  • Ability to assess scenarios holistically.

Responsibilities

  • Review evaluation tasks for logic and completeness.
  • Identify inconsistencies and unclear decision points.
  • Define clear expected behaviors for AI agents.
  • Annotate cause-effect relationships and reasoning paths.
  • Think through complex systems to ensure proper testing.
  • Collaborate with QA, writers, or developers for refinements.

Skills

Analytical thinking
Attention to detail
Communication
Structured data formats understanding

Education

Relevant degree or experience

Job description

A tech consulting firm is seeking an AI Agent Evaluation Analyst for a part-time, flexible role. This opportunity is ideal for individuals with strong analytical and detail-oriented skills who enjoy evaluating complex systems and contributing to advanced AI projects. Candidates will collaborate with developers and QA teams to improve AI agents. A background in consulting, logic puzzles, or policy evaluation is advantageous. Competitive rates may go up to $20/hour.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Agent Evaluation Analyst
AI Agent Evaluation Analyst

Mindrift • Johannesburg

Remote
ZAR 238,959 - 477,918
Flexible remote work
Competitive hourly rates
Experience in advanced AI projects
Remote Part-Time AI Red Team Engineer
Remote Part-Time AI Red Team Engineer

Mindrift • South Africa

Remote
ZAR 462,078 - 658,111
Competitive hourly rates
Remote work flexibility
Experience in cutting-edge AI projects
AI Agent Builder — Hybrid Operations Role
AI Agent Builder — Hybrid Operations Role

Beyond Elevation • Wes-Kaap

Hybrid
ZAR 111,600 - 167,400
Generative AI Evaluator | $30/hr Remote
Generative AI Evaluator | $30/hr Remote

Crossing Hurdles • South Africa

Remote
ZAR 441,892 - 662,838
Remote AI Design Evaluator & Visual Systems Expert
Remote AI Design Evaluator & Visual Systems Expert

Prolific • South Africa

Remote
ZAR 577,000 - 822,000
Remote work
Flexible hours
Remote AI Design Evaluation Specialist
Remote AI Design Evaluation Specialist

Prolific • Cape Town

Remote
ZAR 701,902
Competitive pay rates
Flexible hours
Ability to work from home
AI Coding Agent Evaluator - Design & QA Real-World Tasks
AI Coding Agent Evaluator - Design & QA Real-World Tasks

Mindrift • Durban

Remote
ZAR 740,283 - 1,054,342
Generative AI Quality Evaluator - Remote
Generative AI Quality Evaluator - Remote

Crossing Hurdles • South Africa

Remote
ZAR 441,892 - 662,838
Remote AI Pilot Operations Associate (Part-Time Freelance)
Remote AI Pilot Operations Associate (Part-Time Freelance)

Mindrift • South Africa

Remote
ZAR 183,894
Remote AI Chatbot Tester — Flexible Contract
Remote AI Chatbot Tester — Flexible Contract

Alignerr Corp. • South Africa

Remote
ZAR 83,000 - 220,000