Social Media AI Evaluation Designer

Mercor; Toronto; On-site; CAD 80,000 - 110,000; Full time

Job summary

Mercor is seeking a hands-on Social Media AI Training Environment designer to shape briefs, artifact specifications, and evaluation criteria for real-brand social channels. You will design tasks, define required documents and tools, and establish judgment standards for agent performance.

Ideal candidates bring brand social experience, proficiency with Sprout Social, and a background in AI training environments or simulated case studies, plus a track record of clear reasoning in written

Qualifications

• Hands-on practitioner experience running brand social.
• Fluent in Sprout Social and comfortable reading platform analytics.
• Experience building AI training environments, RL environments or simulated case studies.
• Rubric Academy or Rubric Bootcamp certification preferred.
• Clear written reasoning—explain why a decision is right.

Responsibilities

• Design task briefs drawn from your social media work, highlighting what the task tests.
• Specify documents, dashboards, personas and tool states required for realistic tasks.
• Write or review reference answers and the criteria that separate strong responses from plausible-but-wrong ones.
• Review AI agent attempts and judge them against your own standard.

Skills

Brand social management

Platform analytics literacy

AI training environments

Clear written reasoning

Education

Rubric Academy or Rubric Bootcamp certification

Tools

Sprout Social

Job description

Mercor is seeking a hands-on Social Media AI Training Environment designer to shape briefs, artifact specifications, and evaluation criteria for real-brand social channels. You will design tasks, define required documents and tools, and establish judgment standards for agent performance.

Ideal candidates bring brand social experience, proficiency with Sprout Social, and a background in AI training environments or simulated case studies, plus a track record of clear reasoning in written