AI Evaluation Rubric Auditor

AuraOne Human Data

United States

Remote

USD 83,000 - 117,000

Part time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

AuraOne Human Data is seeking an AI Evaluation Rubric Auditor to remotely review AI outputs across audit workflows. You will re-perform calculations, assess narrative reasoning, and ensure compliance with standards, with citations, severity tags, and corrections documented for training.

This contractor role requires direct audit experience, CPA/CFA/ACA or equivalent, and reliable async availability for at least 10 hours per week.

Qualifications

  • Direct working experience with audit on real engagements, transactions, or audits for AI Evaluation Rubric Auditor work.
  • CPA, CFA, or ACA. A comparable credential or equivalent applied experience also works.
  • Comfort applying multi-page rubrics consistently across long batches.
  • Clear written reasoning that cites standards, regulations, or firm policy.
  • Reliable async availability for at least 10 hours per week.

Responsibilities

  • Review AI outputs against current audit standards, regulations, and firm policy for AI Evaluation Rubric Auditor assignments.
  • Re-perform calculations and flag rounding, classification, or treatment errors.
  • Tag compliance, reconciliation, and disclosure issues with structured severity scores.
  • Capture the corrected workpaper or narrative so the modeling team can train on it.
  • Adjudicate disputed treatments against published standards or firm guidance.
  • Maintain reviewer-quality scores in inter-rater calibration cycles.

Skills

Financial analysis
Audit-grade review
Regulatory standards
Risk assessment
Audit

Education

CPA
CFA
ACA

Job description

AI Evaluation Rubric Auditor is a remote review track for evaluating AI outputs across audit workflows. Reviewers grade calculations, narrative reasoning, and policy adherence; flag compliance and reconciliation issues; and document the correct treatment so the modeling team can train on it.

Why this role matters

Audit workflows leave no room for fuzzy reasoning. AuraOne uses experienced finance-and-risk professionals to grade AI outputs the way an audit reviewer would - with citations, severity tags, and the corrected calculation alongside the original.

Responsibilities
  • Review AI outputs against current audit standards, regulations, and firm policy for AI Evaluation Rubric Auditor assignments.
  • Re-perform calculations and flag rounding, classification, or treatment errors.
  • Tag compliance, reconciliation, and disclosure issues with structured severity scores.
  • Capture the corrected workpaper or narrative so the modeling team can train on it.
  • Adjudicate disputed treatments against published standards or firm guidance.
  • Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
  • Direct working experience with audit on real engagements, transactions, or audits for AI Evaluation Rubric Auditor work.
  • A CPA, CFA, or ACA. A comparable credential or equivalent applied experience also works.
  • Comfort applying multi-page rubrics consistently across long batches.
  • Clear written reasoning that cites standards, regulations, or firm policy.
  • Reliable async availability for at least 10 hours per week.
Example tasks
  • Re-perform an audit calculation produced by a model and flag any rounding or classification errors.
  • Grade a model's narrative reasoning on a complex transaction and rate the citation quality.
  • Adjudicate a disputed treatment between two reviewers using current standards.
  • Audit a 25-row batch for rubric consistency and report drift to the program lead.
Nice to have
  • Big Four, bulge-bracket, or in-house controller / risk experience.
  • Familiarity with AI-assisted close, audit, or risk tooling and its failure modes.
  • Bilingual experience for cross-jurisdiction reviews.
Skills
  • Financial analysis
  • Audit-grade review
  • Regulatory standards
  • Risk assessment
  • Audit
Work model

Remote - US-eligible. Remote - Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.

Compensation

$60-$85 / hr

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Personal Account Rater - Finance & Audit Review
Personal Account Rater - Finance & Audit Review

AuraOne Human Data • United States

Remote
USD 41,000 - 69,000
Finance operations / audit support Evaluator
Finance operations / audit support Evaluator

AuraOne, Inc. • United States

Remote
USD 110,000 - 165,000
Finance Business Analyst, AI Evaluation
Finance Business Analyst, AI Evaluation

AuraOne Human Data • United States

Remote
USD 55,000 - 96,000
Compliance / regulatory response with financial-services AI Evaluator
Compliance / regulatory response with financial-services AI Evaluator

AuraOne, Inc. • United States

Remote
USD 110,000 - 165,000
US remote eligibility
Contractor engagement
Senior Accounting & Finance Advisors - AI Evaluation
Senior Accounting & Finance Advisors - AI Evaluation

AuraOne Human Data • United States

Remote
USD 55,000 - 83,000
Accountant / Auditor
Accountant / Auditor

Auraone • United States

Remote
USD 96,000 - 124,000
AI Evaluation Rubric Reviewer
AI Evaluation Rubric Reviewer

AuraOne Human Data • United States

Remote
USD 55,000 - 103,000
Personal Account Generalist Raters - US
Personal Account Generalist Raters - US

AuraOne Human Data • United States

Remote
USD 34,000 - 55,000
Big Four/industry experience welcome
Familiarity with AI-assisted close/r ض
Bilingual cross-jurisdiction reviews
Remote Finance & Research Analyst
Remote Finance & Research Analyst

AuraOne Human Data • United States

Remote
USD 48,000 - 83,000
Remote Securities & Financial Regulation Expert - AI Review
Remote Securities & Financial Regulation Expert - AI Review

AuraOne Human Data • United States

Remote
USD 83,000 - 138,000