AI Agent Power User - Remote

YO AI Labs; Warszawa; Remote; PLN 323,000 - 485,000; Part time

Job summary

YO AI Labs is seeking experienced AI Agent Power Users to contribute to an enterprise AI training project focused on evaluating and improving AI assistants in real-world business workflows. You will test multi-step processes, document findings, and provide actionable feedback to product teams while leveraging tools like ChatGPT and Claude.

The role is contract-based and remote across the US, Canada, UK, Ireland, Australia, and New Zealand.

Qualifications

• 5+ years in a business/tech role solving operational problems.
• Daily, hands-on use of AI tools like ChatGPT and Claude.
• Experience creating or applying evaluation rubrics or guidelines.
• Comfortable with AI tools and common productivity platforms.
• Excellent written English and strong documentation skills.
• Authorized for contract work and located in the US, Canada, UK, Ireland, Australia or New Zealand.

Responsibilities

• Run complex, multi-step business workflows using AI assistants.
• Document workflows and identify where AI meets or misses business needs.
• Evaluate AI outputs using defined rubrics and quality criteria.
• Provide clear, actionable feedback to AI teams.
• Identify recurring AI strengths, weaknesses and improvement opportunities.
• Connect AI tools with platforms like Google Drive, Gmail, Slack, and Notion.
• Maintain detailed, reproducible documentation of findings.

Skills

AI tools usage

Evaluation rubrics

QA scorecards

Documentation skills

Education

Bachelor's degree or higher

Job description

AI Agent Power User

Role Type: Contractor
Location: Remote - US, Canada, UK, Ireland, Australia, New Zealand

We are seeking experienced AI Agent Power Users to contribute to an enterprise AI training project focused on evaluating and improving AI assistants in real-world business workflows.

You'll use your professional experience and hands-on knowledge of AI tools to test complex scenarios, evaluate outputs, and provide actionable feedback. No formal AI background is required.

Scope of Work

• Run complex, multi-step business workflows using AI assistants such as ChatGPT and Claude.
• Document workflows and identify where AI performance meets or falls short of real business requirements.
• Evaluate AI-generated outputs using defined rubrics and quality criteria.
• Provide clear, actionable feedback to AI research and product teams.
• Identify recurring AI strengths, weaknesses, and opportunities for process improvement.
• Connect AI tools with workplace platforms such as Google Drive, Gmail, Slack, and Notion.
• Maintain detailed documentation to ensure findings are clear, consistent, and reproducible.

Preferred Qualifications

• 5+ years of professional experience in any business function involving technology and operational problem-solving.
• Bachelor's degree or higher in any discipline.
• Daily, hands-on experience using ChatGPT and/or Claude for real professional work.
• Experience creating or applying evaluation rubrics, QA scorecards, content review guidelines, or similar frameworks.
• Comfortable working with AI tools and common workplace productivity platforms.
• Excellent written English and strong documentation skills.
• Authorized for contract work and located in the US, Canada, UK, Ireland, Australia, or New Zealand.