Remote AI Trainer — Software Engineer for Model Evaluation

MissionHires; United States; Remote; USD 55,000 - 172,000; Part time

Benefits offered by this job

Fully remote contract

Flexible scheduling

PayPal payments

Minimum 10 hours/week

Job summary

MissionHires is seeking a versatile software engineering evaluator for a fully remote contract role. You will design challenging tasks for AI coding agents, run them against real codebases, and judge outputs using rubrics, tests, and clear criteria.

You should have 2+ years of professional software engineering experience, an in-progress Bachelor's degree, comfort with Git, tests, debuggers, and strong English. Familiarity with Claude or ChatGPT is a plus.

Qualifications

• 2+ years of professional software engineering experience; hands-on IC work, not just management.
• In-progress Bachelor's degree or higher.
• Ability to use a terminal, Git, tests, and debuggers; quick to understand unfamiliar codebases.
• Strong engineering judgment and ability to describe good code and model failures in writing.
• Full professional English (spoken and written).
• Familiarity with AI tools (Claude or ChatGPT); able to distinguish well-reasoned answers from plausible but incorrect ones.
• Docker and CI experience are a plus.
• CS degree is a plus but not required.

Responsibilities

• Design challenging, realistic software engineering tasks based on professional experience.
• Run tasks through AI coding agents and evaluate their deliverables.
• Compare multiple agent outputs and document strengths/weaknesses.
• Create tests, rubrics, and success criteria.
• Write evidence-based evaluations of failures and edge cases.
• Contribute expertise across the stack and review tasks created by others.
• Propose and build task scenarios from experience, including failure modes.

Skills

English fluency

Software engineering

Independent contributor

Education

In-progress Bachelor's degree

Computer science degree a plus

Tools

Git

Docker

CI tools

Debugger usage

Job description

MissionHires is seeking a versatile software engineering evaluator for a fully remote contract role. You will design challenging tasks for AI coding agents, run them against real codebases, and judge outputs using rubrics, tests, and clear criteria.

You should have 2+ years of professional software engineering experience, an in-progress Bachelor's degree, comfort with Git, tests, debuggers, and strong English. Familiarity with Claude or ChatGPT is a plus.