Software Engineer (AI Trainer)

MissionHires; United States; Remote; USD 55,000 - 172,000; Part time

Benefits offered by this job

Fully remote contract

Flexible scheduling

PayPal payments

Minimum 10 hours/week

Job summary

MissionHires is seeking a versatile software engineering evaluator for a fully remote contract role. You will design challenging tasks for AI coding agents, run them against real codebases, and judge outputs using rubrics, tests, and clear criteria.

You should have 2+ years of professional software engineering experience, an in-progress Bachelor's degree, comfort with Git, tests, debuggers, and strong English. Familiarity with Claude or ChatGPT is a plus.

Qualifications

• 2+ years of professional software engineering experience; hands-on IC work, not just management.
• In-progress Bachelor's degree or higher.
• Ability to use a terminal, Git, tests, and debuggers; quick to understand unfamiliar codebases.
• Strong engineering judgment and ability to describe good code and model failures in writing.
• Full professional English (spoken and written).
• Familiarity with AI tools (Claude or ChatGPT); able to distinguish well-reasoned answers from plausible but incorrect ones.
• Docker and CI experience are a plus.
• CS degree is a plus but not required.

Responsibilities

• Design challenging, realistic software engineering tasks based on professional experience.
• Run tasks through AI coding agents and evaluate their deliverables.
• Compare multiple agent outputs and document strengths/weaknesses.
• Create tests, rubrics, and success criteria.
• Write evidence-based evaluations of failures and edge cases.
• Contribute expertise across the stack and review tasks created by others.
• Propose and build task scenarios from experience, including failure modes.

Skills

English fluency

Software engineering

Independent contributor

Education

In-progress Bachelor's degree

Computer science degree a plus

Tools

Git

Docker

CI tools

Debugger usage

Job description

When AI gets the details wrong in your field, real people pay for it. The people who know the work best have to be the ones training these models. You’re not being replaced by AI. You’re making sure it’s used accurately and safely.

How You'll Contribute

• Design challenging, realistic software engineering tasks based on professional experience, including scenarios, specifications, codebases, tests, logs, API contracts, and design notes.
• Run tasks through AI coding agents and evaluate their deliverables, including code diffs, tests, designs, and pull request reviews.
• Compare multiple agent outputs against identical specifications and codebases, determine which is stronger, and document shortcomings.
• Create tests, rubrics, and success criteria that define correct behavior, edge‑case handling, test coverage, and adherence to conventions.
• Write clear evidence‑based evaluations identifying failures such as brittle changes, missed edge cases, misleading tests, fabricated files, trace thrashing, and off‑spec interpretations.
• Contribute expertise across your primary stack and adjacent technologies, and review or refine tasks created by other engineers.
• Propose and build task scenarios from professional engineering experience, including expected model failure modes.

What Makes You a Great Fit

• 2+ years of professional software engineering experience preferred, including shipping and maintaining production code as part of a team.
• In‑progress Bachelor's degree or higher.
• Hands‑on individual‑contributor engineering experience; candidates should currently or recently have performed the work themselves rather than working solely in management.
• Ability to use a terminal, Git, tests, and debuggers, and to quickly understand unfamiliar codebases.
• Strong engineering judgment and the ability to explain in writing what good code and genuine model failures look like.
• Full professional or native‑level written and spoken English.
• General familiarity with AI and LLM tools, such as Claude or ChatGPT, and the ability to distinguish well‑reasoned answers from plausible but incorrect ones.
• No prior AI or machine learning experience is required.
• Docker and CI experience are a plus.
• A computer science degree is a plus but not required.
• Pay range of $40 to $125 per hour in USD.
• Fully remote contract role with flexible scheduling.
• Minimum commitment of 10 hours per week, with no weekly maximum and full‑time hours available.
• Multi‑day task timers that allow work to be spread across several days.
• Access to Claude and ChatGPT through the project; no personal subscription is required.
• Regular payment via PayPal.
• Opportunity to help improve frontier AI systems using real‑world software engineering expertise.