We are seeking experienced product designers, UX/UI designers, design system specialist, rapid prototypers, and AI-native product builders to create realistic evaluation tasks for advanced AI systems.
What You'll Do
- Analyze task execution frame by frame when necessary
- Identify important actions, events, behaviors, and outcomes
- Apply structured labels, tags, and grades based on project guidelines
- Mark timestamps or video segments corresponding to key events
- Determine whether tasks were completed correctly
- Identify partial completions, failed actions, mistakes, or unusual behavior
- Document observations and evaluation rationales clearly
- Apply annotation guidelines consistently across large volumes of videos
- Participate in reviewer calibration and quality discussions
- Collaborate asynchronously with project trainers and other contributors
- Flag unclear or ambiguous cases for additional review
- Provide feedback on labeling guidelines and annotation workflows
- Organize and submit completed annotations within project timelines
- Maintain high standards for precision, consistency, and quality
Who Can Apply
Relevant backgrounds include:
- Product Designers
- Senior Product Designers
- Staff Product Designers
- Principal Product Designers
- Lead Product Designers
- UX Designers
- UI Designers
- UX/UI Designers
- Interaction Designers
- Experience Designers
- Digital Product Designers
- Visual Designers
- Interface Designers
- Web Designers
- Mobile Product Designers
- and Application Designers
Design leadership backgrounds may include:
- Design Leads
- Design Managers
- Product Design Managers
- Heads of Design
- Design Directors
- UX Directors
- Creative Directors
- Experience Design Directors
- Design Principals
- and Design Consultants
Design systems backgrounds may include:
- Design Systems Designers
- Design Systems Leads
- Design Systems Engineers
- UI Systems Designers
- Component Library Designers
- Design Technologists
- UX Engineers
- Design Engineers
- Frontend Design Engineers
- and Design Infrastructure Specialists
Video and media backgrounds may include:
- Video Editors
- Video Producers
- Multimedia Editors
- Content Editors
- Motion Designers
- Motion Graphics Artists
- Filmmakers
- Videographers
- Cinematographers
- Post-Production Specialists
- and Audiovisual Professionals
Research and usability backgrounds may include:
- UX Researchers
- User Researchers
- Product Researchers
- Usability Researchers
- Human Factors Specialists
- Human-Computer Interaction Professionals
- HCI Researchers
- UX Strategists
- Service Designers
- Information Architects
- and Customer Experience Designers
Annotation and AI data backgrounds may include:
- Video Annotators
- Data Annotators
- Data Labelers
- AI Data Trainers
- AI Evaluators
- Computer Vision Annotators
- Image Annotators
- Multimedia Annotators
- Human Feedback Specialists
- Annotation Quality Analysts
- and Data Quality Reviewers
Requirements
- Professional or academic background in UX/UI, product design, video editing, photography, multimedia, engineering, data annotation, or a closely related field
- Strong visual attention to detail
- Ability to analyze video content carefully
- Comfortable applying detailed annotation and grading guidelines
- Ability to work consistently across repetitive review tasks
- Ability to explain annotation decisions clearly
- Comfortable learning new annotation tools
- Strong organizational skills
- Ability to meet deadlines
- Comfortable collaborating with remote teams
- Ability to work independently while maintaining high quality
Preferred Background
- Experience at a product-led technology company, design-forward organization, startup, or independent product studio
- Experience building and shipping products using AI coding assistants
- Experience moving rapidly from concept to functional prototype
- Strong understanding of both UX principles and technical implementation
- Experience reviewing or critiquing the work of designers or engineers
A formal credential is not required. A strong portfolio of shipped work, real product ownership, and demonstrated senior-level judgment are highly valued.
This opportunity is ideal for experienced product builders who understand both the craft of creating excellent user experiences and the rapidly evolving workflows of AI-assisted product development. We are a referral partner of the client.