AI Evaluation Specialist

SaidGig; New York (NY); Remote; USD 28,000 - 83,000; Full time

Job summary

SaidGig is seeking a remote contractor to develop challenging Q&A content that tests advanced AI models. You will conduct independent research, triangulate information from multiple sources, and generate multi-step questions that require synthesis and analytical reasoning.

Candidates should document their methods with clear citations, refine content per reviewer feedback, and apply domain knowledge to produce high-quality data for training and evaluating next‑generation AI systems.

Qualifications

• Strong ability to conduct thorough, independent research and critically evaluate information from multiple sources.
• Excellent source-triangulation skills and careful citation of sources.
• Exceptional attention to detail and written precision.
• Fluency in written English; English does not need to be your first language.
• Strong analytical thinking and ability to craft nuanced, well-structured questions.
• Self-direction and accountability when working remotely.
• Interest in advanced AI and testing model capabilities.
• Experience in research/writing demonstrated through academic or professional work.
• Recent graduates or advanced-degree holders welcome; diverse backgrounds considered.
• Experience in AI training or content creation is a plus but not required.

Responsibilities

• Develop original, challenging Q&A pairs to test AI models across diverse subjects.
• Conduct in-depth research and triangulate information across multiple sources.
• Create multi-step questions requiring synthesis and analytical reasoning.
• Test questions, analyze performance, and revise content for appropriate difficulty.
• Document research methods with citations and explanations.
• Refine content based on reviewer feedback while aligning with guidelines.
• Apply subject-matter knowledge to produce high-quality input for AI training and evaluation.

Skills

Research skills

Source triangulation

Attention to detail

Academic writing

English fluency

Independent work

Analytical thinking

AI evaluation interest

Job description

Key Responsibilities

• Develop original, challenging question-and-answer pairs that test advanced AI models across diverse subjects.
• Conduct in-depth research and triangulate information across multiple sources to produce accurate, comprehensive, and well-documented answers.
• Create multi-step questions that require synthesis and analytical reasoning rather than simple single-source lookups.
• Test questions against AI models, analyze the resulting performance, and revise content to increase or adjust difficulty.
• Document research methods with clear citations and logical explanations supporting each answer.
• Refine content based on reviewer feedback while maintaining alignment with project guidelines and quality standards.
• Apply subject-matter knowledge and research expertise to produce high-quality input used to train and evaluate next-generation AI systems.

Qualifications

• Demonstrated ability to conduct thorough, independent research and critically evaluate information from multiple sources.
• Strong research and source-triangulation skills.
• Exceptional attention to detail and written precision.
• Fluency in written English; English does not need to be your first language.
• Strong analytical thinking and the ability to craft nuanced, well-structured questions that probe deep understanding.
• Self-direction, reliability, and accountability when working independently in a remote environment.
• Interest in advanced AI and in testing the limits of current model capabilities.
• Research and writing expertise demonstrated through academic or professional work.
• Recent graduates, advanced-degree holders, and candidates from any academic or professional background are welcome to apply.
• Prior experience with AI training, AI evaluation, or content creation is a plus but is not required.

Work Terms

Remote contractor engagement.

Work is completed independently and is task-based.

Minimum submission requirements apply, including a required minimum number of tasks per week.

The time needed to complete individual tasks may vary based on experience and workflow.

Roles are typically filled quickly, with candidates expected to be ready to begin promptly.

If selected, candidates are expected to begin their first tasks within 24 to 48 hours after completing onboarding.

Engagement and access to project work are subject to selection and current project needs.

Compensation

Advertised compensation range: $20 to $60 per hour. Actual compensation is output-based, with payment issued per task that meets project specifications.