Remote AI Data Evaluation Engineer – Code Review & Testing

AuraOne Human Data

United States

Remote

USD 68,000 - 97,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

AuraOne Human Data is hiring a Senior Software Engineer - AI Data Evaluation to remotely review production code and AI outputs for correctness against real-world standards. You will reproduce failures, write unit tests, and explain fixes to help the modeling team target gaps.

You will collaborate with the engineering team to ensure high-quality data evaluation and maintain rigorous review rubrics, with a flexible remote schedule for US-based work.

Qualifications

  • Strong day-job engineering experience reading, running, and debugging unfamiliar code.
  • Ability to write concise unit tests that capture a single failure mode.
  • Familiarity with a testing framework (pytest, Jest, JUnit, etc.).
  • Clear written reasoning that convinces another senior engineer.

Responsibilities

  • Run and reproduce candidate code outputs in a sandboxed environment.
  • Grade AI data evaluation solutions for correctness, style, and edge-case handling.
  • Write minimal failing tests that demonstrate model bugs.
  • Compare paired solutions and provide rationale tied to rubric.
  • Tag failure modes with severity scores (compile error, runtime, etc.).
  • Document recurring code-generation failures for the modeling team.
  • Calibrate against gold-standard reviews to maintain inter-rater consistency.

Skills

Code review
Debugging
Unit testing
Software engineering judgment
AI data engineering review
Software engineering
AI model evaluation

Job description

AuraOne Human Data is hiring a Senior Software Engineer - AI Data Evaluation to remotely review production code and AI outputs for correctness against real-world standards. You will reproduce failures, write unit tests, and explain fixes to help the modeling team target gaps.

You will collaborate with the engineering team to ensure high-quality data evaluation and maintain rigorous review rubrics, with a flexible remote schedule for US-based work.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Code Review Engineer — Remote (US-Based)
AI Code Review Engineer — Remote (US-Based)

AuraOne Human Data • United States

Remote
USD 83,000 - 152,000
Remote Senior AI Data Review Engineer
Remote Senior AI Data Review Engineer

AuraOne, Inc. • United States

Remote
USD 141,000 - 190,000
Senior Software Engineer, AI Data QA & Review (Remote)
Senior Software Engineer, AI Data QA & Review (Remote)

AuraOne Human Data • United States

Remote
USD 170,000 - 243,000
Remote AI Evaluation Engineer — Civil Code Reviewer
Remote AI Evaluation Engineer — Civil Code Reviewer

AuraOne Human Data • United States

Remote
USD 102,000 - 146,000
Remote DevOps Code Review Engineer - AI Evaluator
Remote DevOps Code Review Engineer - AI Evaluator

AuraOne Human Data • United States

Remote
USD 83,000 - 138,000
Remote Software AI Engineer — Code Review & Debugging
Remote Software AI Engineer — Code Review & Debugging

Auraone • United States

Remote
USD 148,000 - 210,000
Senior AI Data Review Engineer — Remote Contractor
Senior AI Data Review Engineer — Remote Contractor

AuraOne Human Data • United States

Remote
USD 68,000 - 97,000
Remote Java Data Engineer - AI Code-Review Specialist
Remote Java Data Engineer - AI Code-Review Specialist

AuraOne Human Data • United States

Remote
USD 55,000 - 110,000
Remote AI Code Review Engineer - DevOps & Infra
Remote AI Code Review Engineer - DevOps & Infra

AuraOne Human Data • United States

Remote
USD 110,000 - 165,000
Senior AI Security Engineer & Code Reviewer (Remote)
Senior AI Security Engineer & Code Reviewer (Remote)

AuraOne Human Data • United States

Remote
USD 83,000 - 152,000