Retail AI Evaluation Specialist

Mercor; New York (NY); Remote; USD 110,000 - 170,000; Full time

Job summary

Cincinnatus LLC, the employer of record, is seeking a seasoned Retail SME to help train AI models by evaluating outputs against rubrics and applying real-world retail judgment.

The role requires 8+ years in retail and the ability to commit at least 35 hours per week, with placement at a leading AI Lab as part of their extended workforce. Strong communication and cross-functional collaboration are essential.

Qualifications

• 8+ years of retail experience at top-tier organizations.
• Hands-on experience evaluating AI model outputs against rubrics.
• Proven ability to communicate clearly and work with cross-functional teams.
• Able to commit at least 35 hours per week on weekdays.

Responsibilities

• Guide teams to close knowledge gaps in retail merchandising, category management, and operations reasoning.
• Design domain-relevant retail tasks and write accurate, well-reasoned solutions grounded in real retail practice.
• Evaluate AI model outputs against rubrics and provide clear written feedback on correctness and reasoning.
• Develop and refine evaluation guidelines and scoring rubrics for retail tasks.
• Collaborate with subject matter experts to ensure consistency and accuracy in training data.

Skills

Retail domain expertise

AI model evaluation

Written & verbal communication

Part-time/Full-time availability 35h

Job description

Cincinnatus LLC, the employer of record, is seeking a seasoned Retail SME to help train AI models by evaluating outputs against rubrics and applying real-world retail judgment.

The role requires 8+ years in retail and the ability to commit at least 35 hours per week, with placement at a leading AI Lab as part of their extended workforce. Strong communication and cross-functional collaboration are essential.