Gmail & Google Calendar AI Assistant Evaluator

Remotebridge; Northern (KY); Remote; USD 60,000 - 90,000; Part time

Job summary

Frontier AI Lab is seeking a Gmail & Google Calendar AI Assistant Evaluator to work remotely as a contractor in the United States. You will perform everyday digital tasks, give clear AI instructions, and review results for accuracy. The role focuses on evaluating AI assistants with structured feedback and detailed grading guidelines.

You will handle emails, invites, scheduling, and coordination scenarios, and contribute to project milestones with precise data submissions and quality reviews.

Qualifications

• Based in the US and able to work remotely on a contract basis.
• Daily use of Gmail and Google Calendar to manage personal tasks.
• Experience coordinating with others via email or text for meetups or plans.
• Comfort booking online (restaurants, travel, appointments, deliveries).
• Experience using AI tools for real-world tasks and validating outputs.
• Strong attention to detail and clear written reporting.

Responsibilities

• Complete realistic digital tasks (emailing, scheduling, booking, coordinating) using AI tools under standardized conditions.
• Provide clear instructions to AI assistants and review results for accuracy and completeness.
• Apply grading guidelines to evaluate, rank, and annotate assistant responses with feedback.
• Test scenarios involving emails, calendar invites, bookings, and multi-person coordination.
• Record findings and submit evaluation data per project milestones.
• Collaborate with trainers to resolve ambiguities, refine guidelines, and ensure consistency.
• Participate in ongoing quality reviews to maintain rigorous evaluation standards.

Skills

US-based

Gmail proficiency

Google Calendar

Attention to detail

Written communication

Time management

AI tools experience

iPhone/iMessage

Social accounts (Facebook/Instagram)

Remote/independent work

Tools

AI evaluation tools

Data labeling platforms

Job description

# Gmail & Google Calendar AI Assistant EvaluatorFrontier AI Lab · Remote · Full-time## Ready to see if you fit?Upload a resume once, then run an honest fit check on this role in seconds. Access is free and always will be.Sign up to check your fit## About the roleRole Title: Gmail & Google Calendar AI Assistant EvaluatorRole Type: Contractor, RemoteLocation: United States onlyOur team is engaging Gmail & Google Calendar AI Assistant Evaluators to collaborate on a customer-driven project enhancing AI assistant quality through real-world task evaluation. In this role, you'll use your everyday experience managing your digital life — email, calendars, bookings, and coordination — to help train and evaluate next-generation AI assistants. Your work will shape how AI models handle the real tasks people delegate every day.Key Responsibilities:• Complete realistic, everyday digital tasks (emailing, scheduling, booking, coordinating) using AI assistant tools under standardized, repeatable conditions.• Give an AI assistant clear instructions for real-world tasks, then carefully review and verify its work for accuracy and completeness.• Apply detailed grading guidelines to evaluate, rank, and annotate assistant responses, providing structured feedback that improves model performance.• Test scenarios involving email threads, calendar invites, online bookings, and multi-person coordination, ensuring coverage of diverse situations and edge cases.• Utilize AI training tools and platforms to record findings and submit evaluation data in alignment with project standards and milestones.• Collaborate with project trainers and contributors to resolve ambiguities, refine guidelines, and ensure evaluation consistency across the team.• Participate in ongoing quality reviews, incorporating feedback to maintain rigorous evaluation standards.Required Skills and Qualifications:• Are based in the US, have an iPhone with iMessage, and have a Facebook or Instagram account.• Daily, hands-on use of Gmail and Google Calendar to run your own life: sending emails, accepting invites, scheduling, and rearranging plans.• Regular experience coordinating with others over email or text — setting up meetups, following up on threads, or confirming shared plans and payments.• Comfort booking things online independently, such as restaurants, appointments, travel, or deliveries.• Experience using AI tools for real, practical tasks, with comfort giving an AI assistant instructions and checking its work.• Exceptional attention to detail and accuracy when reviewing AI-generated output.• Strong written communication skills for clear reporting and timely updates on progress and challenges.• Demonstrated time management and self-organization for independent, remote project participation.### Preferred Qualifications• Prior experience with AI evaluation, data labeling, user testing, or similar feedback-driven projects.• Familiarity with prompt writing and a practical understanding of AI assistant behaviors and limitations.• Experience grading or reviewing AI outputs for accuracy, helpfulness, and safety.