AI Training & Evaluation Lead (Remote)

YO IT Consulting; Auckland; On-site; NZD 70,000 - 110,000; Part time

Job summary

YO IT Consulting is seeking an AI Trainer Lead to remotely support the training of next-generation AI systems. You will evaluate AI responses, annotate large datasets, and provide precise feedback to improve model performance and reliability.

The role focuses on applying detailed rubrics across diverse domains, communicating findings clearly, and collaborating with client teams to refine evaluation criteria. A Bachelor’s degree is required, and the position is remote.

Qualifications

• Bachelor’s degree is required.
• Experience evaluating or scoring large volumes of content against detailed criteria.

Responsibilities

• Evaluate and score AI-generated responses using rubrics and metrics.
• Annotate and categorize data sets to improve model accuracy.
• Review content for relevance, coherence, and factual accuracy with keen attention to detail.
• Provide clear, actionable feedback on AI performance for model improvements.
• Collaborate with the customer’s team to interpret guidelines and refine rubrics.
• Document findings and maintain meticulous records of annotations and evaluations.
• Perform critical reading and analysis of diverse topics from business to legal domains.

Skills

Attention to detail

Critical reading

Communication skills

Independent work

Remote collaboration

Education

Bachelor’s degree

Job description

YO IT Consulting is seeking an AI Trainer Lead to remotely support the training of next-generation AI systems. You will evaluate AI responses, annotate large datasets, and provide precise feedback to improve model performance and reliability.

The role focuses on applying detailed rubrics across diverse domains, communicating findings clearly, and collaborating with client teams to refine evaluation criteria. A Bachelor’s degree is required, and the position is remote.