Research Evaluation Specialist (PhD or Masters / Researcher / Professor)

Confidential; United States; A distancia; USD 55.000 - 83.000; A tiempo parcial

Descripción de la vacante

Zekai is seeking a Research Evaluation Specialist (PhD or Masters) for a remote, world-wide contractor role (~15 hours per week). You will craft high-difficulty QA pairs to rigorously evaluate AI models, source primary references, and document reasoning with citations.

Strong background in AI training or evaluation is required, along with excellent English writing and independent remote work capability. Preferred applicants hold an advanced degree and have a proven track record in scholarly

Formación

• Masters or PhD completed or active candidacy in a research discipline.
• Strong ability to source and triangulate information from primary literature.
• Excellent written communication in English and meticulousness.

Responsabilidades

• Create high-difficulty Q&A pairs to test AI model evaluation.
• Source, verify, and document answers with citations and reasoning.
• Design questions requiring advanced reasoning and cross-source synthesis.
• Test items in AI systems and iteratively raise complexity.
• Write questions and answers with precision and clarity, reducing ambiguity.

Conocimientos

Research expertise

Analytical thinking

English writing

Source primary literature

Independent remote work

Educación

Masters or PhD completed

Active candidacy or equivalent research experience

Descripción del empleo

Role Title: Research Evaluation Specialist (PhD or Masters / Researcher)

Role Type: Contractor (~15 hs a week)

Location: Remote, worldwide

our partner is engaging Research Evaluation Specialists (PhD or Masters / Researcher) to contribute expert insight in developing rigorous test material for advanced AI model evaluation as part of a customer's project. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. Prior experience in AI is required while your domain knowledge brings high value.

Scope of Work

• Create original, high-difficulty question-and-answer pairs at the cutting edge of your discipline to rigorously assess AI model capabilities.
• Source, verify, and thoroughly document answers using primary sources and authoritative references, including clear citations and reasoning.
• Design questions demanding advanced reasoning, methodological nuance, or synthesis across multiple sources, ensuring resistance to oversimplified or shortcut solutions.
• Test questions against AI systems, identify items that are not sufficiently challenging, and iteratively increase complexity while maintaining accuracy.
• Write each question and answer with high precision and clarity, eliminating ambiguity and supporting defensible responses.
• Integrate reviewer feedback and consistently uphold all project guidelines and quality standards throughout your deliverables.

Preferred Qualifications

• Masters or PhD completed, active candidacy, or equivalent research experience as a specialist, researcher, assistant professor or professor in your domain.
• Demonstrated track record of scholarly research, in-depth subject matter expertise, and familiarity with primary literature.
• Exceptional analytical thinking, close attention to detail, and strong written communication skills in English.
• Experience sourcing and triangulating information from authoritative references.
• Proven self-direction and reliability in delivering expert-level work independently within remote engagements.
• Skill in composing challenging, original, and methodologically sound questions.
• Prior experience in AI training, data labeling, or model evaluation is required.

Compensation Structure

Compensation is output-based; experts are paid per task that meets the project specifications. The time required to complete work may vary depending on the expert’s experience and workflow. Minimum submission requirements apply. Experts must submit a minimum of tasks per week.

Start Timeline & Availability

We typically fill roles within 48 hours and are looking for experts ready to jump in right away. If selected, we expect you to start your first tasks within 24-48 hours of completing onboarding.

This is a 100% remote contract role, open to applicants worldwide in the countries the employer accepts.