AI Content Quality Engineer: Evaluation & Frameworks

Instagram; San Francisco (CA); On-site; USD 162,000 - 227,000; Full time

Job summary

Meta's Product Content Engineering group seeks a Content Engineer to advance AI-driven content experiences. You will define quality standards, design evaluation frameworks, and partner with product, engineering, research, and data science to strengthen AI outputs across Meta's apps.

The role involves building repeatable evaluation pipelines, assessing model behavior, and translating findings into concrete recommendations that guide product decisions.

Qualifications

• 5+ years collaborating with product, engineering, design, and user research teams.
• 1+ years with generative AI products, AI evaluation, annotation, and/or content labeling.
• Experience designing evaluation frameworks, annotation guidelines, or quality rubrics for AI/ML systems.
• Demonstrated data analysis skills, with experience exploring data, identifying patterns, and producing actionable insights.
• Experience building new products or platform/ecosystem products.
• Strong cross-functional collaboration and communicating to executive leadership.

Responsibilities

• Define content quality standards and use them to evaluate AI models across products and content experiences.
• Design golden sets, taxonomies, and guidelines for consistent content quality assessments.
• Build workflows for collecting, annotating, and analyzing AI outputs to enable efficient evaluations.
• Evaluate model releases through structured comparison and document improvements and priorities.
• Design evaluation frameworks that integrate qualitative and quantitative signals for user trust and relevance.
• Develop processes to track content quality and model performance over time and flag regressions.
• Synthesize evaluation results into actionable recommendations for engineering and product teams.
• Collaborate cross-functionally with engineers, data scientists, product managers, and content strategists.

Skills

Cross-functional collaboration

Data analysis

AI evaluation

Prompt engineering

Editorial judgment

Communication to leadership

Education

BA/BS in Computer Science, Data Science, Linguistics, or related field

Tools

Python

SQL

Job description

Meta's Product Content Engineering group seeks a Content Engineer to advance AI-driven content experiences. You will define quality standards, design evaluation frameworks, and partner with product, engineering, research, and data science to strengthen AI outputs across Meta's apps.

The role involves building repeatable evaluation pipelines, assessing model behavior, and translating findings into concrete recommendations that guide product decisions.