Senior AI Coding Quality Evaluator (Remote)

G2i Inc.; United States; Remote; USD 138,000 - 276,000; Part time

Job summary

G2i Inc. seeks a highly experienced software engineer (SR+) to evaluate the quality of interactions with modern coding agents such as OpenAI Codex and Claude Code.

This is not a traditional coding role; you won’t write production code but will assess whether the model responds like a great engineer in real-world scenarios. You will judge usefulness, reasoning quality, and engineering judgment, providing clear, opinionated feedback.

Qualifications

• Senior, Staff or Principal-level engineer (or equivalent experience).
• Strong background in TypeScript/JavaScript or Python.
• Hands-on experience with at least one of OpenAI Codex, Claude Code or Cursor.
• Deep familiarity with modern AI-assisted dev workflows.
• Able to evaluate code without executing it or reviewing every line.
• Strong written and spoken English (B2 or above).
• Comfortable giving direct, opinionated feedback.
• High bar for what good engineering looks like.

Responsibilities

• Evaluate AI-generated coding interactions end to end.
• Judge whether outputs are useful, correct at a high level, and aligned with how a strong engineer would think.
• Assess the quality of explanations and reasoning, not just the code.
• Distinguish between levels of response quality (what makes something a 2 vs a 4).
• Give clear, opinionated feedback in writing on what worked, what didn't, and what felt off or misleading.
• Help define what great looks like when working with tools like Cursor, Codex and Claude Code.

Skills

Senior/Staff/Principal level

Strong English writing

Engineering judgment

Experience evaluating AI tools

Tools

OpenAI Codex

Claude Code

Cursor

Job description

G2i Inc. seeks a highly experienced software engineer (SR+) to evaluate the quality of interactions with modern coding agents such as OpenAI Codex and Claude Code.

This is not a traditional coding role; you won’t write production code but will assess whether the model responds like a great engineer in real-world scenarios. You will judge usefulness, reasoning quality, and engineering judgment, providing clear, opinionated feedback.