Accounting Quality & AI Scoring Analyst

Mercor; San Francisco (CA); Remote; USD 110,000 - 150,000; Full time

Job summary

Mercor, in collaboration with a leading AI research organization, seeks experienced accountants to design evaluation criteria and scoring for real-world accounting work. You will craft task-specific grading rubrics for reconciliations, close packages, financial statements and disclosures, and memos, then justify scores in writing.

This role emphasizes rigorous, reproducible judgments, with feedback from senior reviewers and rapid iteration to refine criteria and scoring approaches.

Qualifications

• 5+ years of professional accounting experience.
• Background in audit, assurance, or advisory at leading firms, or senior accounting at large public companies.
• CPA or equivalent certification strongly preferred.
• US GAAP, month-end close, reconciliations, disclosures, and audit-ready workpapers.
• Strong written communication and ability to justify scoring.
• Detail-oriented and able to calibrate judgments with peers.
• Experience with AI training or human-data projects is a plus.

Responsibilities

• Design precise, task-specific grading criteria for real-world accounting deliverables (reconciliations, close packages, financial statements and disclosures, workpapers, technical accounting memos).
• Score AI-generated and human work samples with detailed written justifications for every score.
• Apply consistent, evidence-based judgment so that scores are reproducible and defensible.
• Incorporate structured feedback from senior reviewers and iterate quickly on your work.

Skills

5+ years accounting experience

Education

CPA or equivalent

Job description

Mercor, in collaboration with a leading AI research organization, seeks experienced accountants to design evaluation criteria and scoring for real-world accounting work. You will craft task-specific grading rubrics for reconciliations, close packages, financial statements and disclosures, and memos, then justify scores in writing.

This role emphasizes rigorous, reproducible judgments, with feedback from senior reviewers and rapid iteration to refine criteria and scoring approaches.