Remote AI Task Auditor & SWE Trainer (Freelance)

Lever, Inc.; Österreich; Remote; EUR 62.000 - 84.000; Teilzeit

Benefits dieser Stelle

Fully remote

Flexible projects

Impact on AI training

Competitive pay

Zusammenfassung

Lever, Inc. is seeking a SWE-Bench AI Task Auditor for a freelance AI trainer project based in Austria.

You will review engineering tasks used to train and evaluate advanced AI models, ensuring technical accuracy, realism, reproducibility, and proper testing. You will investigate codebase integrations, test failures, logic issues, and other technical challenges, applying practical software engineering judgment rather than following fixed checks.

Qualifikationen

• Demonstrable professional software engineering experience.
• Strong analytical and problem-solving abilities.
• Ability to navigate complex codebases and debugging tasks.

Aufgaben

• Evaluate SWE-Bench tasks for technical accuracy, realism, solvability, reproducibility, and alignment with SWE-Bench standards.
• Review task codebases, integrations, tests, and evaluation criteria to identify weaknesses.
• Rigorously test and troubleshoot complex technical scenarios to ensure tasks function as intended.
• Investigate codebase integration problems, test failures, and logic or implementation issues.
• Provide clear, actionable feedback to task creators to fix identified problems.
• Apply professional software engineering judgment to assess realism of development scenarios.
• Help maintain high technical rigor across AI training and evaluation workflows.

Kenntnisse

Software engineering

Jobbeschreibung

Lever, Inc. is seeking a SWE-Bench AI Task Auditor for a freelance AI trainer project based in Austria.

You will review engineering tasks used to train and evaluate advanced AI models, ensuring technical accuracy, realism, reproducibility, and proper testing. You will investigate codebase integrations, test failures, logic issues, and other technical challenges, applying practical software engineering judgment rather than following fixed checks.