Generative AI Quality Evaluator - Remote

Crossing Hurdles; South Africa; Remote; ZAR 441,892 - 662,838; Part time

Job summary

A consulting firm is seeking a candidate for a remote role focused on evaluating outputs from large language models. Responsibilities include reviewing workflows, providing feedback, and ensuring model improvements. Applicants should have strong experience in AI output analysis, and keen attention to detail, alongside proficient English communication skills. This position allows for flexible hours ranging from 10 to 40 hours per week, making it suitable for independent workers who can adapt to evolving guidelines.

Qualifications

• Strong experience in LLM evaluation, AI output analysis, or analytical roles.
• Proficiency in rubric-based scoring and AI quality assessment.
• Excellent attention to detail with strong decision-making skills.
• Proficient in English communication, both written and verbal.
• Ability to work independently in a remote environment.

Responsibilities

• Evaluate outputs from large language models using defined quality standards.
• Review multi-step agent workflows to assess accuracy and completeness.
• Provide structured, actionable feedback to support model refinement.
• Document findings clearly and communicate insights to stakeholders.

Skills

LLM evaluation

AI output analysis

QA/testing

UX research

Attention to detail

Independent work

English communication

Job description

A consulting firm is seeking a candidate for a remote role focused on evaluating outputs from large language models. Responsibilities include reviewing workflows, providing feedback, and ensuring model improvements. Applicants should have strong experience in AI output analysis, and keen attention to detail, alongside proficient English communication skills. This position allows for flexible hours ranging from 10 to 40 hours per week, making it suitable for independent workers who can adapt to evolving guidelines.