AI Workflow Evaluator & Personal Task Trainer

Obsidian; New York (NY); On-site; USD 60,000 - 90,000; Full time

Job summary

Obsidian is seeking advanced LLM power users to evaluate AI systems on real-world personal-life workflows. You will use MCP and plugins/connectors like Google Drive, Notion, and Expedia to perform multi-step tasks such as travel planning, health research, home services, and career planning.

The role requires documenting decisions, recording your screen, and articulating why AI outputs are strong, weak, or unrealistic.

Qualifications

• US-based only.
• Strong MCP experience and plug-in/connector usage.
• Experience using LLM plugins/connectors such as Google Drive, Expedia, Notion, and similar tools, multiple times a week.
• Heavy personal usage of LLM products.
• An active, rich LLM account with regular usage and approximately 6+ months of history.
• Willingness to sign a data-share consent form via DocuSign.
• Experience using AI for multi-step planning, research, decision-making, or personal workflows.
• Strong written judgment and attention to detail.
• Ability to explain what makes an AI output good, bad, incomplete, unsafe, or unrealistic.
• Experience writing and evaluating against rubrics.

Responsibilities

• Evaluate AI systems on complex personal workflows and real-world tasks.
• Create realistic prompts for multi-step life tasks and record your screen during execution.
• Use personal plugins/connectors to complete actions and document outcomes.
• Write clear explanations of AI successes and failures and assess practicality and personalization.
• Identify gaps where models miss context or misapply tools and develop rubrics for evaluation.

Skills

MCP experience

Plugin usage

LLM productivity

Tools

Google Drive

Notion

Expedia

Job description

Obsidian is seeking advanced LLM power users to evaluate AI systems on real-world personal-life workflows. You will use MCP and plugins/connectors like Google Drive, Notion, and Expedia to perform multi-step tasks such as travel planning, health research, home services, and career planning.

The role requires documenting decisions, recording your screen, and articulating why AI outputs are strong, weak, or unrealistic.