Senior AI Coding Interaction Evaluator

G2i Inc.; Puerto Rico; On-site; USD 69,000 - 276,000; Full time

Job summary

G2i Inc. is seeking a Senior AI Interaction Evaluator to assess the quality of interactions with modern coding agents such as OpenAI Codex and Claude Code. This is a contract role and you will not be writing production code.

You’ll evaluate AI-generated coding interactions for usefulness, correctness at a high level, and alignment with how a strong engineer would think, focusing on engineering taste over syntax. Expect subjective yet rigorous judgments with direct feedback.

Qualifications

• Strong background in Python and AI-assisted developer workflows.
• Experience evaluating AI coding agents without full code execution.
• Ability to give direct, opinionated feedback.
• Comfort with subjective yet rigorous judgments.
• Experience mentoring engineers or defining engineering standards.

Responsibilities

• Evaluate AI-generated coding interactions end-to-end.
• Judge usefulness and correctness at a high level.
• Assess explanations and reasoning quality.
• Distinguish between different levels of response quality.
• Provide clear, opinionated feedback on what worked and what didn’t.
• Help define what great looks like when interacting with tools like Cursor.

Skills

Python

OpenAI Codex

Cursor

AI-assisted dev workflows

Engineering judgment

Tools

Cursor IDE

Job description

G2i Inc. is seeking a Senior AI Interaction Evaluator to assess the quality of interactions with modern coding agents such as OpenAI Codex and Claude Code. This is a contract role and you will not be writing production code.

You’ll evaluate AI-generated coding interactions for usefulness, correctness at a high level, and alignment with how a strong engineer would think, focusing on engineering taste over syntax. Expect subjective yet rigorous judgments with direct feedback.