Senior AI Coding Evaluation Engineer - Remote, 10-20h/wk

Remote Jobs; United States; Remote; USD 138,000 - 276,000; Part time

Job summary

Remote Jobs is seeking a highly experienced software engineer to evaluate AI-assisted coding interactions. You will assess whether responses make sense, whether the preamble and reasoning are useful, and whether output shows strong engineering judgment.

The role focuses on evaluating quality over writing production code, with feedback delivered in writing and via Loom walkthroughs. Strong English and opinionated feedback are required for success.

Qualifications

• Senior, Staff or Principal-level engineer.
• Hands-on experience with OpenAI Codex, Claude Code or Cursor.
• Deep familiarity with modern AI-assisted dev workflows.
• Strong written and spoken English (B2 or above).

Responsibilities

• Evaluate AI-generated coding interactions end to end.
• Judge whether outputs are useful, correct at a high level, and aligned with how a strong engineer would think.
• Assess the quality of explanations and reasoning, not just the code.
• Distinguish between levels of response quality (2 vs 4).
• Give clear, opinionated feedback in writing on what worked, what didn't, and what felt off or misleading.

Skills

TypeScript/JavaScript

Python

English (B2+)

Job description

Remote Jobs is seeking a highly experienced software engineer to evaluate AI-assisted coding interactions. You will assess whether responses make sense, whether the preamble and reasoning are useful, and whether output shows strong engineering judgment.

The role focuses on evaluating quality over writing production code, with feedback delivered in writing and via Loom walkthroughs. Strong English and opinionated feedback are required for success.