Senior AI Coding Interactions Evaluator

g2i; United States; On-site; USD 138,000 - 276,000; Full time

Job summary

g2i is seeking a Senior AI Interaction Evaluator to assess how AI coding agents behave in real-world scenarios. You will evaluate responses for usefulness, correctness at a high level, and whether the reasoning reflects expert engineering thinking.

This role emphasizes engineering taste over syntax, requires subjective yet rigorous judgments, and includes a take-home exercise plus one behavioral interview. Start ASAP with about 10–20 hours per week, contracting through early May, with the

Qualifications

• Staff / Principal‑level engineer (or equivalent experience).
• Strong background in TS/JavaScript or Python.
• Hands-on experience with OpenAI Codex, Claude Code and Cursor.

Responsibilities

• Evaluate AI-generated coding interactions end-to-end.
• Judge whether outputs are useful, correct (high level) and aligned with strong engineering thinking.
• Assess the quality of explanations and reasoning, not just code.
• Distinguish between levels of response quality (e.g. 2 vs 4).
• Provide clear, opinionated feedback on what worked, what didn’t and what felt off.
• Help define what great looks like when interacting with Cursor.

Skills

Staff / Principal level engineer

TypeScript / JavaScript

Python

Tools

OpenAI Codex

Claude Code

Cursor

Job description

g2i is seeking a Senior AI Interaction Evaluator to assess how AI coding agents behave in real-world scenarios. You will evaluate responses for usefulness, correctness at a high level, and whether the reasoning reflects expert engineering thinking.

This role emphasizes engineering taste over syntax, requires subjective yet rigorous judgments, and includes a take-home exercise plus one behavioral interview. Start ASAP with about 10–20 hours per week, contracting through early May, with the