Shopify Storefront Evaluator - Remote, Part-Time

Mercor; New York (NY); Remote; USD 55,000 - 69,000; Part time

Job summary

Mercor is seeking a contract Shopify Storefront Agent Evaluator working remotely. You will simulate shopper interactions, evaluate agent responses, and provide concise feedback to improve performance. The role requires clear written English and consistent, careful judgment.

Ideal candidates have hands-on Shopify experience and familiarity with AI shopping assistants. Up to 5 hours per week, compensation $40–$50 per hour, remote work.

Qualifications

• Consumer e-commerce fluency and regular online shopping experience.
• Apply rubric uniformly to responses for consistent judgments.
• Articulate concise reasons for any response failures in writing.

Responsibilities

• Evaluate the Storefront Agent by simulating real shopper interactions (discovery, comparisons, sizing, stock, shipping, returns).
• Grade responses on Relevance, Accuracy, and Safety to meet shopper expectations and store policies.
• Provide concise feedback on failures to inform team members not present during evaluation.
• Work independently to deliver precise, repeatable judgments on agent performance.

Skills

Consumer e-commerce fluency

Careful, consistent judgment

Clear, concise written English

Job description

Mercor is seeking a contract Shopify Storefront Agent Evaluator working remotely. You will simulate shopper interactions, evaluate agent responses, and provide concise feedback to improve performance. The role requires clear written English and consistent, careful judgment.

Ideal candidates have hands-on Shopify experience and familiarity with AI shopping assistants. Up to 5 hours per week, compensation $40–$50 per hour, remote work.