AI Personal Assistant Evaluator - Real-World Task Specialist

Mercor; San Francisco (CA); On-site; USD 50,000 - 90,000; Part time

Job summary

Mercor, a leading AI research organization in the San Francisco area, is seeking advanced LLM power users to evaluate how AI systems handle personalized, real-world life tasks. You will judge usefulness, personalization, realism, and success across food, health, productivity, careers, and learning.

This role requires heavy personal use of AI tools and clear written judgments about what makes an AI output good or bad.

Qualifications

• Extensive experience using LLMs for personal workflows.
• Ability to judge usefulness, personalization and realism of AI outputs.
• Strong written communication and attention to detail.
• Experience explaining AI outputs and safety concerns.

Responsibilities

• Evaluate AI responses for usefulness and alignment with user goals.
• Provide clear explanations of what makes outputs good or bad.
• Collaborate with researchers to improve future AI assistants.

Skills

Heavy AI usage

Multi-step tasks

AI tools familiarity

Explain AI outputs

Attention to detail

Tools

ChatGPT

Claude

Gemini

Perplexity

Cursor

Windsurf

Codex

Job description

Mercor, a leading AI research organization in the San Francisco area, is seeking advanced LLM power users to evaluate how AI systems handle personalized, real-world life tasks. You will judge usefulness, personalization, realism, and success across food, health, productivity, careers, and learning.

This role requires heavy personal use of AI tools and clear written judgments about what makes an AI output good or bad.