AI Evaluation Engineer (Remote, Part-Time)

mercor; United States; Remote; USD 83,000 - 103,000; Full time

Job summary

Mercor is seeking a Software & Data Science Expert for a remote contract role. You will design tasks to test AI reasoning over technical documentation, and assess how well AI follows precise instructions and produces accurate outputs.

The position requires collaboration with AI research teams to ensure task relevance and accuracy, while working independently to meet deadlines and drive model improvements. Expect 15–20 hours per week at a rate of $60–$75 per hour.

Qualifications

• 3+ years of experience in software engineering or data science & analytics.

Responsibilities

• Author tasks to test AI reasoning over technical documentation.
• Evaluate AI ability to follow precise instructions and produce accurate outputs.
• Develop objective evaluation rubrics for task assessment.
• Collaborate with AI research teams to ensure task relevance and accuracy.
• Work independently and asynchronously to meet deadlines and improve AI model performance.

Skills

Software engineering

Data science

Analytics

Job description

Mercor is seeking a Software & Data Science Expert for a remote contract role. You will design tasks to test AI reasoning over technical documentation, and assess how well AI follows precise instructions and produces accurate outputs.

The position requires collaboration with AI research teams to ensure task relevance and accuracy, while working independently to meet deadlines and drive model improvements. Expect 15–20 hours per week at a rate of $60–$75 per hour.