Remote QA/Test Engineer - AI Benchmarking (Contract)

mercor

United States

Remote

USD 83,000 - 124,000

Part time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Mercor is a San Francisco–headquartered company that connects elite creative and technical talent with leading AI research labs. We are seeking a QA/Test Engineer on a contract basis for remote work, with a commitment of 35 hours per week and compensation of $60–$90 per hour.

The role involves designing test cases, reviewing tasks for ambiguity, debugging with Python, and contributing to quality checklists to ensure reliable AI agent runs. Strong documentation is essential.

Qualifications

  • MSc or PhD in a STEM field or equivalent practical experience.
  • 1+ years in test engineering, QA, or research/software engineering.
  • Design test cases, quality-review processes, and debugging complex systems.
  • Proficiency in Python and Git.
  • Exceptional attention to detail and clear documentation habits.

Responsibilities

  • Design test cases to confirm each task works as intended, including tricky edge cases.
  • Review tasks and reference solutions to catch ambiguity and gaps early.
  • Debug using Python when tasks or checks don't behave as expected.
  • Shape quality checklists and provide actionable feedback to authors.
  • Protect benchmark scores by identifying shortcuts and grading gaps in AI agent runs.

Skills

Design test cases
Quality assurance
Debugging complex systems
Documentation habits

Education

MSc or PhD in STEM field
Equivalent practical experience

Tools

Python
Git

Job description

Mercor is a San Francisco–headquartered company that connects elite creative and technical talent with leading AI research labs. We are seeking a QA/Test Engineer on a contract basis for remote work, with a commitment of 35 hours per week and compensation of $60–$90 per hour.

The role involves designing test cases, reviewing tasks for ambiguity, debugging with Python, and contributing to quality checklists to ensure reliable AI agent runs. Strong documentation is essential.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote QA/Test Engineer for AI Benchmarking & Debugging
Remote QA/Test Engineer for AI Benchmarking & Debugging

mercor • United States

Remote
USD 83,000 - 124,000
QA/Test Engineer - Fully Remote | Upto $90/hr
QA/Test Engineer - Fully Remote | Upto $90/hr

mercor • United States

Remote
USD 83,000 - 124,000
Test Engineer - Fully Remote | Upto $90/hr
Test Engineer - Fully Remote | Upto $90/hr

mercor • United States

Remote
USD 83,000 - 124,000
Part-Time QA Engineer (AI Web Apps & Automation)
Part-Time QA Engineer (AI Web Apps & Automation)

Mercor • United States

Remote
USD 34,000 - 55,000
Senior AI/ML Software Engineer (Remote Contract)
Senior AI/ML Software Engineer (Remote Contract)

Mercor • San Francisco (CA)

Hybrid
GBP 123,000 - 185,000
Remote QA Engineer: Browser Test Validation for AI Apps
Remote QA Engineer: Browser Test Validation for AI Apps

mercor • United States

Remote
USD 41,000 - 83,000
Remote QA Engineer & Browser Test Reviewer (Contract)
Remote QA Engineer & Browser Test Reviewer (Contract)

mercor • United States

Remote
USD 41,000 - 83,000
QA Engineer | Remote - Contract
QA Engineer | Remote - Contract

Xperteez Technology • United States

Remote
USD 34,000 - 69,000
Remote QA Engineer – AI Platform & Test Automation
Remote QA Engineer – AI Platform & Test Automation

WebSenor Ltd • United States

Remote
USD 70,000 - 110,000
QA Test Engineer- Remote
QA Test Engineer- Remote

YO AI Labs • New York (NY)

Remote
USD 34,000 - 69,000