Catalog and Metadata Evaluator, 2581-1

Cypress HCM; Raleigh (NC); On-site; USD 96,432 - 103,320; Part time

Job summary

A leading media analytics company based in Raleigh, NC, is seeking a Catalog & Metadata Evaluator. This role involves evaluating and annotating content for AI training to enhance the user experience. The ideal candidate should have a strong passion for films and global entertainment, possess expert judgment in content analysis, and operate effectively in a detail-oriented environment. The position offers a pay rate of $70-$75/hr, making it an exciting opportunity to improve member-facing experiences through high-quality data.

Qualifications

• Prior experience in media as a writer, editor, content designer, or metadata analyst.
• Passionate about films and series with knowledge of the U.S. entertainment industry.
• Ability to communicate findings in an objective, concise manner.

Responsibilities

• Rate, annotate, and review content recommendations for LLM training.
• Apply expert judgment to capture nuance and intent within content.
• Collaborate with internal partners to improve dataset quality.

Skills

Deep knowledge of visual media

Expert judgment

Detail-oriented

Fluency in English

Job description

Overview

If you are an avid fan and a serious consumer of visual entertainment, the Member Experience team at the client is seeking a unique candidate who can capture the essence of films, series, and specials, all while keeping the experience of our users top of mind. As a Catalog & Metadata Evaluator, your work will directly contribute to the training, evaluation, and improvement of GenAI applications that power member-facing experiences. In this role, you will apply expert judgment to rate, annotate, and review large-scale content datasets used to train and refine AI systems. By bringing nuance, consistency, and clarity to content evaluation, you will help ensure that AI-driven solutions better understand films and series, respond accurately to member needs, and deliver high-quality, trusted experiences at scale.

The Ideal Candidate

• Has a deep and broad knowledge of visual media and is passionate about global entertainment, with a strong understanding of films and series across genres and markets. They can succinctly and objectively communicate what is important about a piece of content—and why—distilling complex creative works into clear, structured insights.
• The ideal candidate is experienced as an expert annotator, rater, or dataset reviewer, particularly in structured or rubric-based environments that support AI or machine-learning use cases. They are comfortable applying expert judgment within defined frameworks, navigating nuance and ambiguity while maintaining consistency and quality at scale.
• They thrive in a high-volume, high-quality, deadline- and data-driven environment, and are highly detail-oriented, reliable, and disciplined in their work. They are flexible, adaptive, and curious, with the ability to evolve alongside changing tools, processes, and organizational needs, and are motivated by work that directly improves member-facing experiences through high-quality data.

Responsibilities

• Rate, annotate, and review content recommendations and generated text to support training, evaluation, and quality improvement of large language models (LLMs)
• Apply established guidelines and expert judgment to capture nuance, intent, and context within creative content
• Review and validate datasets used in AI and machine-learning workflows for accuracy, consistency, and completeness
• Identify gaps, edge cases, or patterns in annotations that may impact model performance or member-facing outcomes
• Document evaluations and rationale in a clear, objective, and concise manner
• Collaborate with internal partners across product, data, and AI teams to continuously improve dataset quality and relevance

Qualifications

• Prior experience in media as a writer, editor, content designer, or metadata analyst OR experience as a domain-expert content labeller and AI trainer.
• Passionate about films and series with a deep knowledge of the U.S. entertainment industry and the global streaming landscape
• Ability to distinguish nuances within different content genres and distill the essence of a film/series, including sharing findings in an objective, concise manner
• Ability to master internal tools and systems successfully
• Fluency, and strong conversational writing skills in English required, other languages a plus.
• Experience working with large data catalogs is a plus

Pay Rate

• $70-75/hr.