AI Evaluation Specialist - Remote Contract

micro1; United States; Remote; AUD 70,000 - 110,000; Full time

Job summary

micro1 is seeking an AI Evaluation Specialist to assess AI outputs for an enterprise training initiative. You will evaluate responses against rubrics, provide concise feedback, and help refine evaluation criteria to improve model quality. Remote contractor role with emphasis on independence and attention to detail.

You will analyze large volumes of examples, identify gaps, and document findings to influence model refinement and AI adoption best practices.

Qualifications

• Experience in grading, QA, editorial review, assessment, annotation, or similar fields requiring careful analysis and detailed feedback.
• Advanced daily use of AI assistants (ChatGPT, Claude, etc.) as a core productivity tool.
• Ability to synthesize complex information and communicate findings clearly in writing.
• Background in process improvement, rubric development, or operational quality assessment in enterprise/education contexts.
• Strong critical thinking with focus on consistency, integrity, and fairness in evaluations.
• Comfort working independently on large volumes of similar examples with high attention to detail.
• Collaborative mindset for sharing insights and refining evaluation criteria as models evolve.

Responsibilities

• Evaluate AI-generated outputs against rubrics and quality standards focusing on accuracy, relevance, and guidelines adherence.
• Apply impartial judgment across a high volume of examples ensuring a fair assessment process.
• Identify reasoning gaps, tool-use failures, or logic errors and provide actionable feedback for model improvement.
• Produce clear written feedback highlighting strengths and areas for improvement.
• Participate in discussions on rubric interpretation and evolving quality standards.
• Maintain meticulous documentation of evaluations and recommendations for transparency and traceability.

Skills

Grading & QA feedback

Daily AI assistant use

Synthesis & writing

Process improvement

Collaborative dispute resolution

Independent work on large volumes

Evaluation criteria refinement

Job description

micro1 is seeking an AI Evaluation Specialist to assess AI outputs for an enterprise training initiative. You will evaluate responses against rubrics, provide concise feedback, and help refine evaluation criteria to improve model quality. Remote contractor role with emphasis on independence and attention to detail.

You will analyze large volumes of examples, identify gaps, and document findings to influence model refinement and AI adoption best practices.