Evaluation Specialist/Recent Grad

Confidential; United States; Remote; USD 48,000 - 69,000; Part time

Job summary

Zekai is seeking Evaluation Specialists/Recent Grads for a remote, contractor role contributing to a high-impact AI project. You will develop QA pairs, conduct source-triangulated research, and craft multi-step questions to probe model understanding.

No prior AI experience required; strong research and writing skills in English are essential. Roles start quickly with output-based compensation and worldwide eligibility.

Qualifications

• Thorough, independent research combined with multi-source triangulation.
• Exceptional attention to detail and written precision in English.
• Ability to craft nuanced, well-structured questions that probe deep understanding.
• Strong self-direction, reliability, and accountability in remote work.
• Interest in cutting-edge AI and testing model capabilities.

Responsibilities

• Develop original, challenging question-and-answer pairs testing AI models.
• Conduct in-depth research with rigorous source triangulation for accuracy.
• Create multi-step questions requiring synthesis and analytical reasoning.
• Test questions against AI models, analyze outcomes, and adjust difficulty.
• Document research process with clear citations and explanations.
• Refine content based on reviewer feedback and project guidelines.

Skills

Independent research

Attention to detail

Written English proficiency

Question crafting

Remote work discipline

AI training/evaluation experience (a +

Job description

Role Title: Evaluation Specialist/Recent Grad

Role Type: Contractor

Location: Remote

our partner is engaging Evaluation Specialists/Recent Grads to contribute to a high-impact project for a customer at the frontier of artificial intelligence. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.

Scope of Work

• Develop original, challenging question-and-answer pairs that test the limits of advanced AI models across diverse subjects.
• Conduct in-depth research, applying rigorous source triangulation to ensure each answer is accurate, comprehensive, and well-documented.
• Create multi-step questions that require synthesis and analytical reasoning — not just single-source lookups.
• Test your questions against AI models, analyze outcomes, and iterate to increase or adjust difficulty as needed.
• Document your research process, providing clear citations and logical explanations for each answer.
• Refine content based on reviewer input and maintain alignment with project guidelines and quality standards.

Preferred Qualifications

• Proven ability to conduct thorough, independent research and critically evaluate information from multiple sources.
• Exceptional attention to detail and written precision in English (fluency required, but it does not need to be your first language).
• Talent for crafting nuanced and well-structured questions that probe deep understanding.
• Strong self-direction, reliability, and accountability in remote, independent project work.
• Interest in cutting-edge AI and a passion for testing the boundaries of current model capabilities.
• Prior experience in AI training, evaluation, or content creation is a plus, but not required.
• Open to recent graduates, advanced-degree holders, and individuals from any academic or professional background with demonstrated research and writing expertise.

Compensation Structure

Compensation is output-based; experts are paid per task that meets the project specifications. The time required to complete work may vary depending on the expert’s experience and workflow. Minimum submission requirements apply. Experts must submit a minimum of tasks per week.

Start Timeline & Availability

We typically fill roles within 48 hours and are looking for experts ready to jump in right away. If selected, we expect you to start your first tasks within 24–48 hours of completing onboarding.

This is a 100% remote contract role, open to applicants worldwide in the countries the employer accepts.