Senior Retail AI Evaluation Lead

Obsidian; New York (NY); Remote; USD 70,000 - 90,000; Full time

Job summary

Cincinnatus LLC is seeking a Retail SME to support an AI training data program for a leading AI lab in the United States. You will judge real-world retail tasks and help ensure quality by evaluating outputs against rubrics, guiding teams, and shaping evaluation guidelines.

Required: 8+ years in retail at top-tier companies, hands-on rubric evaluation of LLMs, and ability to commit 35+ hours per week on weekdays. This is a W-2 role through Cincinnatus, with placement at a premier AI facility.

Qualifications

• 8+ years of dedicated professional experience in retail at top-tier organizations.
• Hands-on experience evaluating LLM/AI model outputs against rubrics or structured scoring criteria.
• Demonstrable career progression (e.g., Category Manager to Director).
• Ability to engage reliably for at least 35 hours/week during weekdays.
• Strong verbal and written communication, problem-solving and interpersonal skills.

Responsibilities

• Guide research and engineering teams to close knowledge gaps in retail merchandising, category management, and operations reasoning.
• Design domain-relevant retail tasks and write accurate, well-reasoned solutions grounded in real retail practice.
• Evaluate AI model outputs against structured rubrics and provide clear feedback on correctness, judgment, and reasoning quality.
• Develop and refine evaluation guidelines and scoring rubrics specific to retail tasks.
• Collaborate with other subject matter experts to ensure consistency and accuracy in training data.

Skills

Retail domain expertise

Rubrics evaluation experience

Communication skills

Problem solving

Job description

Cincinnatus LLC is seeking a Retail SME to support an AI training data program for a leading AI lab in the United States. You will judge real-world retail tasks and help ensure quality by evaluating outputs against rubrics, guiding teams, and shaping evaluation guidelines.

Required: 8+ years in retail at top-tier companies, hands-on rubric evaluation of LLMs, and ability to commit 35+ hours per week on weekdays. This is a W-2 role through Cincinnatus, with placement at a premier AI facility.