AI Evaluation Task Author — Computational Researcher

AfterQuery; New Zealand; Remote; NZD 227,000 - 324,000; Full time

Job summary

AfterQuery is building a network of researchers to author and validate research-grade scientific-computing tasks drawn from your own computational work. You can contribute by designing tasks, setting evaluation metrics, and refining submissions with structured feedback, all on a flexible contractor basis.

Experts receive $200 per approved task (about 2 hours each), with no cap on how many tasks you take on and a standard background/identity verification process for all participants.

Qualifications

• Pursuing or holding a PhD with active hands-on research, or a Master's with a research-active record.
• Working proficiency in Python to run and debug own computational work.
• Able to turn own computational research work into a well-specified evaluation task.
• Able to work independently and manage time in a flexible, task-based role.

Responsibilities

• Design and author original research-grade evaluation tasks based on your own computational work
• Specify the metrics a strong solution should be judged by, without publishing the exact passing thresholds
• Revise tasks based on structured review feedback until they're approved
• Optionally review and calibrate tasks submitted by other experts in your domain

Skills

Python

Education

PhD with active hands-on research

Master's with a research-active record

Job description

AfterQuery is building a network of researchers to author and validate research-grade scientific-computing tasks drawn from your own computational work. You can contribute by designing tasks, setting evaluation metrics, and refining submissions with structured feedback, all on a flexible contractor basis.

Experts receive $200 per approved task (about 2 hours each), with no cap on how many tasks you take on and a standard background/identity verification process for all participants.