AI Code Quality Auditor & Feedback Architect

Parailabs

United States

Remote

USD 120,000 - 190,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Parailabs seeks an evaluator to analyze AI-assisted software-development traces used to train and assess frontier AI models. You will judge correctness, workflow soundness, and reasoning across end-to-end coding sessions generated with AI tools.

Qualified candidates have 3+ years of professional software development, hands-on experience with AI-assisted coding tools, and strong debugging skills across backend or full-stack systems. Prior tooling contributions are a plus.

Qualifications

  • 3+ years of professional software development.
  • Hands-on with AI-assisted coding tools and agentic workflows.
  • Strong code-reading and debugging across full-stack or backend systems.
  • Ability to evaluate multi-step coding trajectories for correctness.

Responsibilities

  • Evaluate the quality and correctness of AI-assisted software-development traces.
  • Assess end-to-end coding sessions produced with AI-assisted developer tools.
  • Provide rubric-based written feedback on accuracy, workflow soundness, and reasoning.

Skills

AI-assisted coding tools
Code reading & debugging
3+ years software development

Tools

Cursor
GitHub Copilot
Claude Code

Job description

Parailabs seeks an evaluator to analyze AI-assisted software-development traces used to train and assess frontier AI models. You will judge correctness, workflow soundness, and reasoning across end-to-end coding sessions generated with AI tools.

Qualified candidates have 3+ years of professional software development, hands-on experience with AI-assisted coding tools, and strong debugging skills across backend or full-stack systems. Prior tooling contributions are a plus.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Developer Trace Task Auditor
AI Developer Trace Task Auditor

Parailabs • United States

Remote
USD 120,000 - 190,000
Senior AI Coding QA Evaluator (Remote Contract)
Senior AI Coding QA Evaluator (Remote Contract)

G2i Inc. • United States

Remote
USD 138,000 - 276,000
AI Benchmark Task Auditor & Quality Reviewer
AI Benchmark Task Auditor & Quality Reviewer

Parailabs • United States

Remote
USD 120,000 - 180,000
Remote AI Code Evaluator & DevOps Engineer
Remote AI Code Evaluator & DevOps Engineer

Turing • United States

Remote
USD 11,000 - 22,000
Fully remote
Freelance contractor role
AI Software Engineer — Code Review & Model Evaluation
AI Software Engineer — Code Review & Model Evaluation

AfterQuery Experts • San Diego (CA)

On-site
USD 100,000 - 140,000
Competitive Pay
AI Code Evaluation Engineer (Contract)
AI Code Evaluation Engineer (Contract)

Turing • Pittsburgh

Remote
USD 55,000 - 96,000
AI Code Evaluation Engineer – Frontend & Backend
AI Code Evaluation Engineer – Frontend & Backend

HumanitApp • San Francisco (CA)

On-site
USD 83,000 - 138,000
Remote AI Coding Evaluator – Software Quality
Remote AI Coding Evaluator – Software Quality

Turing • United States

Remote
USD 3,300 - 5,000
Fully remote
Cutting-edge AI projects
Remote AI Code Quality Engineer
Remote AI Code Quality Engineer

Aston Carter • United States

Remote
USD 45,000 - 60,000
Senior AI Coding Quality Evaluator (Remote)
Senior AI Coding Quality Evaluator (Remote)

G2i • United States

Remote
USD 138,000 - 276,000
Remote, worldwide