Remote AI Reasoning & Evaluation Specialist

mercor; Australia; Remote; AUD 118,000 - 158,000; Full time

Job summary

Mercor connects elite creative and technical talent with leading AI research labs. This contract role is remote and pays $70/hour.

The Generalist Expert will evaluate AI-generated responses to strengthen reasoning and rigor, provide structured feedback to improve data quality, and develop clear, well-evidenced written rationales. You will work independently to meet deadlines while improving model performance.

Qualifications

• Bachelor’s degree from a top-500 globally ranked university preferred.
• Strong analytical and written communication skills.
• Ability to work independently and follow detailed task guidelines.
• Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
• Ability to write clear, precise, and well-evidenced written rationales.
• Consistent and honest judgment, including the ability to give critical assessments.
• Strict attention to detail and accurate application of structured evaluation guidelines.
• Ability to work entirely without AI writing tools.
• Native English fluency required.

Responsibilities

• Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs.
• Provide structured written feedback to improve training data quality and downstream performance.
• Develop clear, precise, and well-evidenced written rationales beyond surface-level observations.
• Apply strict attention to detail and accurate application of structured evaluation guidelines.
• Work independently and asynchronously to meet deadlines while improving AI model performance.

Skills

Analytical thinking

Written communication

Independent work

Critical reading

Clear written rationales

Judgment

Attention to detail

AI-free writing

Native English

Education

Bachelor’s degree (top-500 university)

Job description

Mercor connects elite creative and technical talent with leading AI research labs. This contract role is remote and pays $70/hour.

The Generalist Expert will evaluate AI-generated responses to strengthen reasoning and rigor, provide structured feedback to improve data quality, and develop clear, well-evidenced written rationales. You will work independently to meet deadlines while improving model performance.