Senior Software Engineer, AI Coding Tasks (Contract)

Askable; Brisbane; Remote; AUD 93,677,000 - 104,698,000; Full time

Job summary

Askable is seeking a Senior Software Engineer to craft AI coding tasks on a remote basis. You will transform known repository changes into tasks, write briefs, correct solutions, and robust grading checks.

Tasks are used to train AI models and improve evaluation systems. Requirements include 2+ years of software engineering, proficiency in one of Python/TypeScript/Go/Rust/Java/C++, and strong testing practices.

Qualifications

• 2+ years of professional software engineering
• Strong in at least one language (Python/TypeScript/Go/Rust/Java/C++)
• Docker, Linux, Git and automated testing
• Regular use of an AI coding tool
• Experience writing tests, rubrics or evaluation criteria for AI models
• Ability to explain a change clearly and concisely
• At least 10 hours a week starting now

Responsibilities

• Propose a change from a repository you know and submit a short proposal
• Build the task: set up repo as before the change, write the brief, solution, and grading checks
• Stress-test: ensure checks catch lazy/incorrect/shortcut solutions
• Model testing: run tasks against leading AI models to reveal weaknesses
• Review: undergo multiple rounds of review and revision
• Start new task after acceptance

Skills

2+ years exp

Python

TypeScript

Go

Rust

Java

C++

AI coding tool

Automated testing

Testing rubrics

Tools

Docker

Linux

Git

Job description

Senior Software Engineer, AI Coding Tasks (Contract)

We're looking for experienced software engineers to build AI coding tasks from a codebase they know well. $1,400 AUD per task, and you can do as many tasks as you want.

You turn a real change from a repository you know into a task that today's best AI coding models struggle with: hard for the models, but fair for a skilled engineer. Your tasks are used to measure and train AI coding models.

Remote, any location. $1,400 AUD per task that passes review. Minimum of one task a week, about 10 hours a week.

How it works

Propose. Once you're selected, pick a change from a repository you know and submit a short proposal. We approve it or suggest a better candidate.

Build. In our platform, set up the repository as it was before the change, then write the brief, the correct solution and the grading checks.

Stress-test. Try lazy, incorrect and shortcut solutions and make sure your checks catch them.

Model testing. Run your task against leading AI models until strong models often fail but a skilled engineer can clearly solve it.

Review. A reviewer checks your task and you revise it. Most tasks take more than one round.

Next task. Once your task is accepted, start another.

Requirements

2+ years of professional software engineering

Code merged into a public open-source repository, ideally one you know well. A private codebase you own may also qualify (not employer or client code)

Strong in at least one of Python, TypeScript, Go, Rust, Java or C++

Solid with Docker, Linux, Git and automated testing

Regular use of an AI coding tool

Experience writing tests, rubrics or evaluation criteria, ideally for AI models

You can explain a change clearly and concisely, in your own words

At least 10 hours a week, starting now

A secure personal or work computer (no public or shared machines)

Nice to have

Deep expertise in one area, such as databases, compilers, distributed systems, security or infrastructure

Experience building agent tasks or RL environments

Terms

$1,400 AUD for each task that passes our quality review, paid directly to you

Once your proposal is qualified, you sign Askable's contract and accept its terms

The work is used for AI training. The change and your solution must be your own, and you must be free to license them to us

Don't publish the work anywhere: no public pull request or public fork