Marketing Quality Architect for AI Evaluation

Obsidian; New York (NY); On-site; USD 90,000 - 150,000; Full time

Job summary

Mercor is partnering with a leading AI research organization to design tasks and grade real-world marketing work. You will define what excellent work looks like, write criteria, and provide rigorous written justifications for scores.

Ideal candidates bring 5+ years in marketing or advertising, with experience as a creative director, account lead, or brand strategist at top agencies or leading consumer brands. Strong written communication is essential.

Qualifications

• 5+ years of professional experience in marketing or advertising.
• Background as a creative director, account lead, or brand strategist at top-tier agencies.
• Deep fluency in crafting briefs, campaign development, and brand messaging.
• Excellent written communication with ability to justify scores.
• Detail-oriented with ability to calibrate judgments with peers.

Responsibilities

• Design precise, task-specific grading criteria for real-world marketing deliverables.
• Score AI-generated and human work samples against those criteria with written justifications.
• Apply consistent, evidence-based judgment to ensure reproducible scores.
• Incorporate structured feedback from senior reviewers and iterate quickly.

Skills

Marketing experience

Advertising background

Brand strategy

Creative brief writing

Written communication

Job description

Mercor is partnering with a leading AI research organization to design tasks and grade real-world marketing work. You will define what excellent work looks like, write criteria, and provide rigorous written justifications for scores.

Ideal candidates bring 5+ years in marketing or advertising, with experience as a creative director, account lead, or brand strategist at top agencies or leading consumer brands. Strong written communication is essential.