Senior AI Interaction Evaluator for Coding Agents

G2i; United States; Remote; USD 138,000 - 276,000; Part time

Job summary

G2i is seeking a Senior AI Interaction Evaluator to assess how AI coding agents behave in real-world scenarios and provide actionable feedback. The role focuses on evaluating usefulness, correctness at a high level, and engineering judgment, not writing production code.

You will judge explanations and guidance to ensure interactions feel like they come from a strong engineer. Start ASAP, approximately 10–20 hours per week, through early May with possible extension.

Qualifications

• Staff/Principal-level engineer (or equivalent experience)
• Strong background in TypeScript/JavaScript or Python
• Hands-on experience with AI-assisted development tools and workflows

Responsibilities

• Evaluate AI-generated coding interactions end-to-end
• Judge usefulness, correctness at a high level, and engineering judgment
• Assess explanations, reasoning, and how well the model guides the user
• Differentiate quality levels (2 vs 4) and provide actionable feedback
• Define what great looks like when interacting with tools like Cursor

Skills

Senior-level engineer

TypeScript/JavaScript

Python

Tools

OpenAI Codex

Claude Code

Cursor

Job description

G2i is seeking a Senior AI Interaction Evaluator to assess how AI coding agents behave in real-world scenarios and provide actionable feedback. The role focuses on evaluating usefulness, correctness at a high level, and engineering judgment, not writing production code.

You will judge explanations and guidance to ensure interactions feel like they come from a strong engineer. Start ASAP, approximately 10–20 hours per week, through early May with possible extension.