Remote Shopify Shop App Annotator (E-commerce Tagging)

mercor; United States; Remote; USD 55,000 - 69,000; Part time

Job summary

Mercor is seeking a Shopify Shop App Annotator to evaluate shopper-agent conversations and apply tagging within a fixed taxonomy. This contract, remote role offers about 5 hours per week at $40–$50 per hour.

Responsibilities include evaluating conversations, identifying thumbs-down moments, assigning primary and secondary tags, and writing concise explanations of labeling decisions. Ideal for precise, English-literate annotators with structured-reading skills.

Qualifications

• Apply a fixed taxonomy consistently.
• Comfort reading structured traces to understand agent actions.
• Restraint under ambiguity to use the unknown label appropriately.
• Clear, concise written English for explaining labeling decisions.

Responsibilities

• Evaluate full shopper-agent conversations, including tool calls and their outcomes.
• Identify the specific turn that received a thumbs-down from the shopper.
• Classify the issue with a primary domain tag and a specific secondary tag.
• Write concise comments explaining the shopper's intent, the agent's actions, and the rationale for the tags.
• Work independently and asynchronously to maintain consistency in tagging across conversations.

Skills

Rule discipline

Structured reading

Ambiguity handling

Clear English writing

Job description

Mercor is seeking a Shopify Shop App Annotator to evaluate shopper-agent conversations and apply tagging within a fixed taxonomy. This contract, remote role offers about 5 hours per week at $40–$50 per hour.

Responsibilities include evaluating conversations, identifying thumbs-down moments, assigning primary and secondary tags, and writing concise explanations of labeling decisions. Ideal for precise, English-literate annotators with structured-reading skills.