AI Code Trace Auditor — Quality Feedback

Mercor; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Mercor is seeking a professional to evaluate the quality and correctness of AI-assisted software-development traces used to train frontier AI lab models. You will assess end-to-end coding sessions produced with AI-assisted developer tools, judging correctness, workflow soundness, and reasoning, and provide rubric-based written feedback.

You will collaborate with researchers to ensure evaluative criteria are applied consistently and feedback is clearly documented for tooling improvements.

Qualifications

• 3+ years of professional software development experience.
• Hands-on experience with AI-assisted coding tools and agentic workflows (Cursor, Copilot, Claude Code).
• Strong code-reading and debugging skills across full-stack or backend systems.

Responsibilities

• Evaluate quality and correctness of AI-assisted development traces used to train models.
• Assess end-to-end coding sessions produced with AI-assisted developer tools.
• Judge correctness, workflow soundness and reasoning; provide rubric-based feedback.
• Summarize feedback in clear, actionable reports.

Skills

3+ years software development

Tools

Cursor

GitHub Copilot

Claude Code

Job description

Mercor is seeking a professional to evaluate the quality and correctness of AI-assisted software-development traces used to train frontier AI lab models. You will assess end-to-end coding sessions produced with AI-assisted developer tools, judging correctness, workflow soundness, and reasoning, and provide rubric-based written feedback.

You will collaborate with researchers to ensure evaluative criteria are applied consistently and feedback is clearly documented for tooling improvements.