Senior AI Coding Evaluation Engineer - Remote

Openkrill; United States; Remote; USD 138,000 - 276,000; Part time

Job summary

Openkrill is seeking a senior to staff level software engineer to help evaluate the quality of interactions with modern coding agents. This is not production coding; you will assess whether AI-generated responses show strong engineering judgment.

You will judge usefulness, clarity of explanations, and the model's ability to guide a developer. Experience with TypeScript/JavaScript or Python and hands-on work with Codex, Claude Code, or Cursor are required.

Qualifications

• Senior, Staff or Principal-level engineer (or equivalent experience).
• Strong background in TypeScript/JavaScript or Python.
• Hands-on experience with at least one of OpenAI Codex, Claude Code or Cursor.
• Deep familiarity with modern AI-assisted dev workflows.
• Strong written and spoken English (B2 or above). You'll be writing detailed feedback and recording short video explanations, so this is a hard requirement.
• Comfortable giving direct, opinionated feedback.
• High bar for what good engineering looks like.

Responsibilities

• Evaluate AI-generated coding interactions end to end.
• Judge whether outputs are useful, correct at a high level, and aligned with how a strong engineer would think.
• Assess the quality of explanations and reasoning, not just the code.
• Distinguish between levels of response quality (what makes something a 2 vs a 4).
• Give clear, opinionated feedback in writing on what worked, what didn't, and what felt off or misleading.
• Help define what great looks like when working with tools like Cursor, Codex and Claude Code.

Skills

TypeScript/JavaScript

Python

Strong English communication

Engineering judgment

Tools

OpenAI Codex

Claude Code

Cursor

Job description

Openkrill is seeking a senior to staff level software engineer to help evaluate the quality of interactions with modern coding agents. This is not production coding; you will assess whether AI-generated responses show strong engineering judgment.

You will judge usefulness, clarity of explanations, and the model's ability to guide a developer. Experience with TypeScript/JavaScript or Python and hands-on work with Codex, Claude Code, or Cursor are required.