AI Content Quality Engineer: Evaluation & Frameworks

Meta; San Francisco (CA); On-site; USD 162,000 - 227,000; Full time

Job summary

Meta is seeking a Content Engineer for the AI Discovery team to define how AI content experiences are evaluated and improved. You will work at the intersection of content quality, AI evaluation, and the search/recommendation systems powering Meta's products.

You will design frameworks, run golden sets, and collaborate across engineering, product, research, and data science to drive measurable quality and user trust in AI outputs.

Qualifications

• 5+ years collaborating with product, engineering, design, and user research teams.
• 1+ years working with generative AI products and AI evaluation.
• Experience designing evaluation frameworks and quality rubrics for AI/ML systems.
• Data analysis skills with ability to extract actionable insights.
• Experience building new products or platform/ecosystem products.
• Strong communication to executive leadership and cross-functional teams.

Responsibilities

• Define content quality standards and evaluate AI model performance across products.
• Design golden sets, taxonomies, and guidelines for consistent quality assessments.
• Build repeatable workflows for collecting, annotating, and analyzing AI outputs.
• Evaluate model releases and document improvements, regressions, and priorities.
• Design evaluation frameworks combining qualitative and quantitative signals.
• Track content quality and model performance over time, flag regressions.
• Synthesize results into actionable recommendations for engineering and product teams.
• Collaborate with engineers, data scientists, product managers, and content strategists.

Education

BA/BS in CS/DS/ Linguistics

Tools

Python

SQL

Job description

Meta is seeking a Content Engineer for the AI Discovery team to define how AI content experiences are evaluated and improved. You will work at the intersection of content quality, AI evaluation, and the search/recommendation systems powering Meta's products.

You will design frameworks, run golden sets, and collaborate across engineering, product, research, and data science to drive measurable quality and user trust in AI outputs.