Marketing Quality Architect & Evaluator

Mercor; New York (NY); On-site; USD 85,000 - 120,000; Full time

Job summary

Mercor, in partnership with a leading AI research organization, seeks experienced marketing professionals to define what constitutes excellent work by creating grading criteria for real-world marketing deliverables.

You will score AI-generated and human samples, provide detailed written rationales, and ensure scores are reproducible. This role emphasizes clear storytelling, rigorous analysis, and collaborative feedback in a fast-paced environment.

Qualifications

• 5+ years of professional experience in marketing or advertising.
• Background as creative director, account lead, or brand strategist at top-tier agencies.
• Deep fluency in writing and evaluating briefs and campaigns.
• Exceptionally strong written communication; able to justify judgments clearly.
• Detail-oriented and able to calibrate judgments with peers.
• Prior AI training, evaluation, or human-data project experience is a plus.
• No specialized media-planning or measurement background required.

Responsibilities

• Design precise, task-specific grading criteria for marketing deliverables.
• Score AI-generated and human work samples with detailed written justifications.
• Apply consistent, evidence-based judgment for reproducible scores.
• Incorporate feedback from senior reviewers and iterate quickly.

Skills

Marketing strategy

Creative direction

Brand positioning

Written communication

Attention to detail

AI evaluation experience

Analytical thinking

Job description

Mercor, in partnership with a leading AI research organization, seeks experienced marketing professionals to define what constitutes excellent work by creating grading criteria for real-world marketing deliverables.

You will score AI-generated and human samples, provide detailed written rationales, and ensure scores are reproducible. This role emphasizes clear storytelling, rigorous analysis, and collaborative feedback in a fast-paced environment.