Senior AI Interaction Evaluator for Coding Agents

Doist; Miami (FL); On-site; USD 138,000 - 276,000; Part time

Benefits offered by this job

Competitive pay rate

Flexible schedule

Possible extension

ASAP start

Job summary

Doist is seeking a Senior AI Interaction Evaluator to assess AI coding agents like Codex and Claude Code. You won’t write production code but will judge whether outputs reflect strong engineering judgment and useful reasoning.

The role focuses on subjective yet rigorous evaluation of interactions, with a start ASAP timeline and 10–20 hours per week at a rate of $100–$200 per hour, through early May with possible extension.

Qualifications

• Staff / Principal-level engineer or equivalent experience.
• Strong background in TS/JS and Python.
• Hands-on experience with AI coding assistants (Codex / Claude Code).

Responsibilities

• Evaluate AI-generated coding interactions end-to-end.
• Judge usefulness and correctness at a high level.
• Assess explanations and reasoning quality, not just code.
• Provide direct, opinionated feedback to define high-quality interactions.

Skills

TypeScript/JavaScript

Python

Code evaluation

Engineering judgment

Tools

OpenAI Codex

Claude Code

Cursor

Job description

Doist is seeking a Senior AI Interaction Evaluator to assess AI coding agents like Codex and Claude Code. You won’t write production code but will judge whether outputs reflect strong engineering judgment and useful reasoning.

The role focuses on subjective yet rigorous evaluation of interactions, with a start ASAP timeline and 10–20 hours per week at a rate of $100–$200 per hour, through early May with possible extension.