AI Strategy Evaluation Architect

Mercor; San Francisco (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

Mercor, in partnership with a leading AI research organization in the United States, seeks experienced management consultants to define what excellent work looks like. You will design task-specific grading criteria for market analyses, strategy recommendations, financial models, client-ready slides, and implementation roadmaps.

You will score both AI-generated and human deliverables against these criteria, providing detailed written justifications for every score, and incorporate feedback from

Qualifications

• 5+ years of professional management consulting experience.
• Background at a top-tier strategy or management consulting firm (e.g., McKinsey, Bain, BCG, or equivalent).
• Deep fluency in structured problem solving, market sizing, financial modeling and business-case building, and slide-based executive communication.
• Exceptionally strong written communication.
• Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.
• Prior experience with AI training, evaluation, or human-data projects is a strong plus.

Responsibilities

• Design precise, task-specific grading criteria for real-world consulting deliverables (market analyses, strategy recommendations, financial models, client-ready slide decks, and implementation roadmaps).
• Score AI-generated and human work samples against those criteria, with detailed written justifications for every score
• Apply consistent, evidence-based judgment so that scores are reproducible and defensible
• Incorporate structured feedback from senior reviewers and iterate quickly on your work

Skills

Management consulting

Structured problem solving

Market sizing

Financial modeling

Executive communications

Written communication

Judgment calibration

Job description

Mercor, in partnership with a leading AI research organization in the United States, seeks experienced management consultants to define what excellent work looks like. You will design task-specific grading criteria for market analyses, strategy recommendations, financial models, client-ready slides, and implementation roadmaps.

You will score both AI-generated and human deliverables against these criteria, providing detailed written justifications for every score, and incorporate feedback from