Remote AI UX Design Evaluator (Contract)

mercor; San Francisco (CA); Hybrid; AUD 158,000 - 296,000; Full time

Job summary

Mercor is seeking a UI/UX Design Expert for a remote, contract-based role. You will design grading criteria for real-world digital products and evaluate AI-generated and human work samples against defined criteria.

Apply evidence-based judgment, incorporate senior feedback, and iterate quickly to improve AI model performance. Strong written communication and 5+ years in product design are essential.

Qualifications

• 5+ years of professional experience in product design, UI/UX, or design systems.
• Experience at leading digital-product UX studios or in-house product design teams.
• Deep fluency in interaction and visual design and design systems.
• exceptionally strong written communication skills.

Responsibilities

• Design precise, task-specific grading criteria for real-world digital product and UX deliverables.
• Score AI-generated and human work samples against criteria with detailed written justifications.
• Apply consistent, evidence-based judgment for reproducible scores.
• Incorporate structured feedback from senior reviewers and iterate quickly on work.
• Work independently and asynchronously to evaluate and improve AI model performance.

Skills

UI/UX design

Design systems

Written communication

Detail-oriented

Tools

Figma

Job description

Mercor is seeking a UI/UX Design Expert for a remote, contract-based role. You will design grading criteria for real-world digital products and evaluate AI-generated and human work samples against defined criteria.

Apply evidence-based judgment, incorporate senior feedback, and iterate quickly to improve AI model performance. Strong written communication and 5+ years in product design are essential.