LLM Power User & Real-World AI Evaluator

Obsidian; San Francisco (CA); Remote; USD 34,000 - 83,000; Full time

Job summary

Obsidian, a leading AI research organization, is seeking advanced LLM power users to evaluate how well AI systems handle personalized, real-world life tasks. You will judge usefulness, personalization, and realism of AI responses across daily activities.

Ideal candidates use AI tools heavily in personal life and can explain why outputs are good, bad, or unsafe. This role emphasizes practical judgment, attention to detail, and clear written feedback to improve AI assistants.

Qualifications

• Heavily uses LLM products in daily life.
• Experience using AI for multi-step tasks, planning, research, or personal workflows.
• Familiarity with AI agents such as ChatGPT, Claude, Gemini, Perplexity, Cursor, Windsurf, Codex.

Responsibilities

• Evaluate AI outputs on real-world life tasks.
• Assess usefulness, personalization, realism, and success of AI responses.
• Document criteria and provide clear feedback to improve AI systems.
• Focus on tasks across food, health, productivity, careers, and learning.

Skills

Heavy personal LLM usage

Multi-step task experience

Critical evaluation

Strong written judgment

Tools

ChatGPT

Claude

Gemini

Perplexity

Cursor

Windsurf

Codex

Job description

Obsidian, a leading AI research organization, is seeking advanced LLM power users to evaluate how well AI systems handle personalized, real-world life tasks. You will judge usefulness, personalization, and realism of AI responses across daily activities.

Ideal candidates use AI tools heavily in personal life and can explain why outputs are good, bad, or unsafe. This role emphasizes practical judgment, attention to detail, and clear written feedback to improve AI assistants.