Remote SWE-Bench Task Auditor — AI Training Quality

Agency; United States; Remote; USD 146,026,000 - 198,374,000; Full time

Job summary

Agency is seeking experienced software engineering specialists to audit tasks used for training and evaluating AI systems. You will assess technical accuracy, realism, and solvability, and provide actionable feedback on codebase integration issues and test failures.

Role is a freelance contract with remote work. Pay is $60 per hour, with rate determined after evaluating experience and location.

Qualifications

• Proven experience in software engineering and navigating complex codebases.
• Deep knowledge of SWE-Bench standards and evaluation criteria.
• Strong analytical and problem-solving skills to test and troubleshoot complex technical scenarios.

Responsibilities

• Audit tasks used to train and evaluate AI systems for technical accuracy and realism.
• Provide clear, actionable feedback on codebase integration issues, test failures, or logic errors.

Skills

Software engineering

SWE-Bench standards

Analytical skills

Job description

Agency is seeking experienced software engineering specialists to audit tasks used for training and evaluating AI systems. You will assess technical accuracy, realism, and solvability, and provide actionable feedback on codebase integration issues and test failures.

Role is a freelance contract with remote work. Pay is $60 per hour, with rate determined after evaluating experience and location.