LLM Power User: Real-World Task Evaluator

Obsidian; San Francisco (CA); On-site; USD 55,000 - 96,000; Full time

Job summary

Obsidian in San Francisco is seeking advanced LLM power users to evaluate how well AI systems handle personalized, real-world life tasks. You will judge usefulness, personalization, realism, and safety across diverse domains such as food, health, productivity, careers, and learning.

Expect a commitment of 20-40 hours per week, with flexible scheduling to fit your personal workflows. Your detailed feedback will directly inform improvements to AI assistants, making them more trustworthy and

Qualifications

• Heavy personal usage of LLM products.
• Experience using AI for multi-step tasks, planning, research, decision-making, or personal workflows.
• Familiarity with tools such as ChatGPT, Claude, Gemini, Perplexity, Cursor, Windsurf, Codex, or other AI agents.
• Ability to explain what makes an AI output good, bad, incomplete, unsafe, or unrealistic.
• Strong written judgment and attention to detail.

Responsibilities

• Evaluate how well AI systems handle personalized, real-world tasks.
• Provide clear judgments on usefulness, personalization, realism, and safety of AI outputs.
• Document findings to help improve AI assistants for real-world workflows.

Skills

Heavy personal usage of LLMs

Strategic planning

Strong written judgment

Context personalization awareness

Tools

ChatGPT

Claude

Gemini

Perplexity

Cursor

Codex

Job description

Obsidian in San Francisco is seeking advanced LLM power users to evaluate how well AI systems handle personalized, real-world life tasks. You will judge usefulness, personalization, realism, and safety across diverse domains such as food, health, productivity, careers, and learning.

Expect a commitment of 20-40 hours per week, with flexible scheduling to fit your personal workflows. Your detailed feedback will directly inform improvements to AI assistants, making them more trustworthy and