AI Coding Benchmark Engineer — Remote, $1k per Task

NoDesk; United States; Remote; USD 39,000 - 52,000; Full time

Job summary

NoDesk is seeking experienced software engineers to craft AI coding tasks from repositories you know well. You will propose changes, reconstruct the repo state, write clear task briefs, implement the correct solution with tests, and design robust grading checks.

Tasks are paid $1,000 each once they pass quality review, with a minimum of one task per week and roughly 10 hours weekly. Remote work is available globally.

Qualifications

• 2+ years of professional software engineering
• Code merged into a public open-source repository, ideally one you know well
• Strong in at least one of Python, TypeScript, Go, Rust, Java or C++
• Solid with Docker, Linux, Git and automated testing
• Regular use of an AI coding tool
• Experience writing tests, rubrics or evaluation criteria
• Ability to explain a change clearly and concisely
• At least 10 hours a week, starting now
• A secure personal or work computer (no public or shared machines)

Responsibilities

• Propose changes from repositories you know well
• Set up the repository state as it was before the change
• Write brief task descriptions, the correct solution and grading checks
• Stress-test with lazy, incorrect and shortcut solutions
• Run your task against AI models to ensure solvability by a skilled engineer
• Undergo reviewer checks and revise as needed
• Begin another task after acceptance

Skills

Python

TypeScript

Go

Rust

Java

C++

Tools

Docker

Linux

Git

Automated testing

Job description

NoDesk is seeking experienced software engineers to craft AI coding tasks from repositories you know well. You will propose changes, reconstruct the repo state, write clear task briefs, implement the correct solution with tests, and design robust grading checks.

Tasks are paid $1,000 each once they pass quality review, with a minimum of one task per week and roughly 10 hours weekly. Remote work is available globally.