AI Developer Trace Task Auditor

Parailabs

United States

Remote

USD 120,000 - 190,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Parailabs seeks an evaluator to analyze AI-assisted software-development traces used to train and assess frontier AI models. You will judge correctness, workflow soundness, and reasoning across end-to-end coding sessions generated with AI tools.

Qualified candidates have 3+ years of professional software development, hands-on experience with AI-assisted coding tools, and strong debugging skills across backend or full-stack systems. Prior tooling contributions are a plus.

Qualifications

  • 3+ years of professional software development.
  • Hands-on with AI-assisted coding tools and agentic workflows.
  • Strong code-reading and debugging across full-stack or backend systems.
  • Ability to evaluate multi-step coding trajectories for correctness.

Responsibilities

  • Evaluate the quality and correctness of AI-assisted software-development traces.
  • Assess end-to-end coding sessions produced with AI-assisted developer tools.
  • Provide rubric-based written feedback on accuracy, workflow soundness, and reasoning.

Skills

AI-assisted coding tools
Code reading & debugging
3+ years software development

Tools

Cursor
GitHub Copilot
Claude Code

Job description

Evaluate the quality and correctness of AI-assisted software-development traces used to train and evaluate a frontier AI lab's models. You'll assess end-to-end coding sessions produced with AI-assisted developer tools — judging correctness, workflow soundness, and reasoning — and provide clear, rubric-based written feedback.

Basic Qualifications
  • 3+ years professional software development
  • Hands-on experience with AI-assisted coding tools and agentic / spec-driven workflows (Cursor, GitHub Copilot, Claude Code, or similar)
  • Strong code-reading and debugging skills across full-stack or backend systems
  • Ability to evaluate multi-step coding trajectories for correctness and best practice
Preferred Qualifications
  • Experience with Kiro or Amazon CodeCatalyst
  • Prior work evaluating or grading AI-generated code
  • Contributions to developer tooling
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Code Quality Auditor & Feedback Architect
AI Code Quality Auditor & Feedback Architect

Parailabs • United States

Remote
USD 120,000 - 190,000
AI-Assisted Developer Workflows (Trace) AI Task Auditor - Freelance AI Trainer Project
AI-Assisted Developer Workflows (Trace) AI Task Auditor - Freelance AI Trainer Project

Agency • United States

Remote
USD 55,000 - 96,000
AI Developer Trace Task Auditor
AI Developer Trace Task Auditor

HumanitApp • United States

Hybrid
USD 96,000 - 124,000
AI-Assisted Developer Workflows (Trace) AI Task Auditor
AI-Assisted Developer Workflows (Trace) AI Task Auditor

Meridial • United States

Remote
USD 68,000 - 97,000
AI-Assisted Developer Workflows (Trace) AI Task Auditor - Freelance AI Trainer Project
AI-Assisted Developer Workflows (Trace) AI Task Auditor - Freelance AI Trainer Project

Meridial • Austin (TX)

On-site
USD 50,000 - 83,000
Senior AI Coding QA Evaluator (Remote Contract)
Senior AI Coding QA Evaluator (Remote Contract)

G2i Inc. • United States

Remote
USD 138,000 - 276,000
Coding Tools Engineer (Sr. AI Full-Stack Engineer with Coding Tools [Claude Code, Codex] and agentic AI
Coding Tools Engineer (Sr. AI Full-Stack Engineer with Coding Tools [Claude Code, Codex] and agentic AI

Ampcus Inc • Chantilly (VA)

On-site
USD 120,000 - 190,000
Remote AI-Assisted Developer Workflow Auditor
Remote AI-Assisted Developer Workflow Auditor

Meridial • United States

Remote
USD 68,000 - 97,000
AI-Assisted Developer Workflows (Trace) AI Task Auditor - Freelance AI Trainer Project
AI-Assisted Developer Workflows (Trace) AI Task Auditor - Freelance AI Trainer Project

Invisible Technologies Inc. • United States

Remote
USD 76,000 - 90,000
AI Trainer, Software Engineering Code Review (Remote)
AI Trainer, Software Engineering Code Review (Remote)

AuraOne Human Data • United States

Remote
USD 55,000 - 103,000
Remote work