Remote Bench Task Auditor — Docker, Python, Java

Mercor

United States

Remote

USD 96,000 - 124,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

DigiNo is seeking a software engineering reviewer to assess benchmark tasks used to test AI models. You will verify that code fixes, tests, and grading setup are correct and reproducible across environments.

The ideal candidate has at least three years of professional engineering experience, a history of merged open-source contributions, and strong Python skills plus one other major language. This 100% remote role is US-only and full-time at 40 hours per week.

Qualifications

  • Requires at least three years of professional software engineering experience.
  • Merged open-source contributions encouraged.
  • Strong Python plus one other major language.

Responsibilities

  • Review benchmark tasks used to test AI models and verify code fixes, tests, and grading setup for correctness and reproducibility.

Job description

DigiNo is seeking a software engineering reviewer to assess benchmark tasks used to test AI models. You will verify that code fixes, tests, and grading setup are correct and reproducible across environments.

The ideal candidate has at least three years of professional engineering experience, a history of merged open-source contributions, and strong Python skills plus one other major language. This 100% remote role is US-only and full-time at 40 hours per week.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

SWE-Bench Task Auditor (Docker, Python, Java)
SWE-Bench Task Auditor (Docker, Python, Java)

Mercor • United States

Remote
USD 96,000 - 124,000
AI Benchmark Task Auditor & Quality Reviewer
AI Benchmark Task Auditor & Quality Reviewer

Parailabs • United States

Remote
USD 120,000 - 180,000
AI Benchmark Engineer (Remote Contract)
AI Benchmark Engineer (Remote Contract)

LinkedIn • United States

Remote
USD 83,000 - 165,000
SWE-Bench Task Auditor
SWE-Bench Task Auditor

Parailabs • United States

Remote
USD 120,000 - 180,000
SWE-Bench Task Auditor
SWE-Bench Task Auditor

Marketframe • United States

On-site
USD 140,000 - 190,000
Remote Python Code Reviewer (Contract)
Remote Python Code Reviewer (Contract)

Crossing Hurdles • United States

Remote
MXN 533,000 - 759,000
Remote AI Code Benchmark Engineer
Remote AI Code Benchmark Engineer

Turing • United States

Remote
USD 83,000 - 152,000
Fully remote
Cutting-edge AI projects
Remote Senior Python Engineer - AI Code Review & QA
Remote Senior Python Engineer - AI Code Review & QA

prolificacademicltd • United States

Remote
USD 57,000 - 81,000
Flexible schedule
Remote work
Remote Code Reviewer & Debugger (Go/Python/TS)
Remote Code Reviewer & Debugger (Go/Python/TS)

Auraone • United States

Remote
USD 148,000 - 210,000
Remote work
Remote Big Data Engineer - Code Review & QA Specialist
Remote Big Data Engineer - Code Review & QA Specialist

Auraone • United States

Remote
USD 83,000 - 165,000
Remote work
Hourly contract
US eligibility