SWE-Bench AI Task Auditor

Meridial

United States

Remote

USD 68,000 - 97,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Aitrainergigs seeks experienced SWE specialists to audit SWE-Bench tasks used to train AI systems. You will verify technical accuracy, realism, solvability, and reproducibility, and provide actionable feedback on codebase issues and test failures. This is a remote freelance contract, paying $60 per hour; the rate is set after evaluating your experience and location, with equipment provided by you.

Qualifications

  • Proven expertise in software engineering and SWE-Bench standards.
  • Strong analytical and problem-solving skills to rigorously test, troubleshoot, and evaluate complex technical scenarios.

Responsibilities

  • Audit SWE-Bench tasks for technical accuracy, realism, solvability, and reproducibility.
  • Provide actionable feedback on codebase integration issues, test failures, or logic errors.

Skills

SWE-Bench expertise
Code review
Analytical problem solving

Tools

Git
CI/CD

Job description

Project Overview We are sourcing experienced technical specialists in Software Engineering (SWE-Bench) to audit tasks used to train and evaluate AI systems. The objective of this project is to ensure that AI training workflows and tasks are technically rigorous, practical, and highly accurate within your specific area of expertise.

Project Deliverables & Scope
  • Task Evaluation: Assess whether software engineering tasks are technically accurate, realistic, solvable, reproducible, and supported by reliable tests and evaluation criteria.

  • Technical Auditing & Feedback: Provide clear, actionable feedback on any identified codebase integration issues, test failures, or logic errors within the tasks.

Required Expertise
  • Demonstrable professional experience and deep technical knowledge in software engineering (including complex codebase navigation, real-world application development, and SWE-Bench standards). (Note: Candidates will be considered for one specialty based on their experience; expertise across other domains is not required).

  • Strong analytical and problem-solving skills to rigorously test, troubleshoot, and evaluate complex technical scenarios.

We offer a pay rate of $60 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. As a contractor you’ll supply a secure computer and high-speed internet; company-sponsored benefits such as health insurance and PTO do not apply.

Employment type: Freelance Contract

Workplace type: Remote

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

SWE-Bench AI Task Auditor - Freelance AI Trainer Project
SWE-Bench AI Task Auditor - Freelance AI Trainer Project

Agency • United States

Remote
USD 106,000 - 144,000
Remote SWE-Bench Task Auditor — AI Training Quality
Remote SWE-Bench Task Auditor — AI Training Quality

Agency • United States

Remote
USD 106,000 - 144,000
Remote SWE-Bench Task Auditor (Contract)
Remote SWE-Bench Task Auditor (Contract)

Meridial • United States

Remote
USD 68,000 - 97,000
Technical Task Auditor — SWE-Bench Task Auditor
Technical Task Auditor — SWE-Bench Task Auditor

Meridial • United States

Remote
USD 83,000 - 165,000
AI-Assisted Developer Workflows (Trace) AI Task Auditor
AI-Assisted Developer Workflows (Trace) AI Task Auditor

Meridial • United States

Remote
USD 68,000 - 97,000
Scientific and Technical Service Specialist
Scientific and Technical Service Specialist

Meridial • United States

Remote
USD 14,000 - 41,000
Scientific and Technical Service Specialist - Freelance AI Trainer Project
Scientific and Technical Service Specialist - Freelance AI Trainer Project

Invisible Technologies Inc. • United States

Remote
USD 14,000 - 41,000
AWS Trainium/NKI (TRN) AI Task Auditor
AWS Trainium/NKI (TRN) AI Task Auditor

Meridial • United States

Remote
USD 68,000 - 97,000
AI-Assisted Developer Workflows (Trace) AI Task Auditor - Freelance AI Trainer Project
AI-Assisted Developer Workflows (Trace) AI Task Auditor - Freelance AI Trainer Project

Invisible Technologies Inc. • United States

Remote
USD 76,000 - 90,000
AWS Trainium/NKI (TRN) AI Task Auditor - Freelance AI Trainer Project
AWS Trainium/NKI (TRN) AI Task Auditor - Freelance AI Trainer Project

Agency • United States

Remote
USD 70,000 - 95,000