Product Content Engineer

Meta; San Francisco (CA); On-site; USD 162,000 - 227,000; Full time

Job summary

Meta is seeking a Content Engineer for the AI Discovery team to define how AI content experiences are evaluated and improved. You will work at the intersection of content quality, AI evaluation, and the search/recommendation systems powering Meta's products.

You will design frameworks, run golden sets, and collaborate across engineering, product, research, and data science to drive measurable quality and user trust in AI outputs.

Qualifications

• 5+ years collaborating with product, engineering, design, and user research teams.
• 1+ years working with generative AI products and AI evaluation.
• Experience designing evaluation frameworks and quality rubrics for AI/ML systems.
• Data analysis skills with ability to extract actionable insights.
• Experience building new products or platform/ecosystem products.
• Strong communication to executive leadership and cross-functional teams.

Responsibilities

• Define content quality standards and evaluate AI model performance across products.
• Design golden sets, taxonomies, and guidelines for consistent quality assessments.
• Build repeatable workflows for collecting, annotating, and analyzing AI outputs.
• Evaluate model releases and document improvements, regressions, and priorities.
• Design evaluation frameworks combining qualitative and quantitative signals.
• Track content quality and model performance over time, flag regressions.
• Synthesize results into actionable recommendations for engineering and product teams.
• Collaborate with engineers, data scientists, product managers, and content strategists.

Education

BA/BS in CS/DS/ Linguistics

Tools

Python

SQL

Job description

Summary:

Product Content Engineering is a horizontal function supporting initiatives across Meta's family of apps. We partner closely with product and technical teams to solve problems by providing content-centered solutions, setting standards of quality, and building the frameworks that ensure AI-powered experiences actually work for people. We're looking for a Content Engineer to join our AI Discovery team and help define how Meta evaluates and improves AI content experiences. You'll work at the intersection of content quality, AI evaluation, and the search and recommendation systems that power Meta's products: building the frameworks, rubrics, and pipelines that hold AI outputs to a high standard. You'll assess model behavior, identify where it falls short, and work cross-functionally with engineering, product, research, and data science teams to make it better.If you're energized by the opportunity to build better AI product experiences through rigorous evaluation, have experience navigating ambiguity by defining structure, prioritizing work, and driving clarity in evolving problem spaces, and apply sound editorial and analytical judgment to content quality decisions, we encourage you to apply.

Required Skills:

Product Content Engineer Responsibilities:

1. Define content quality standards and use them to systematically evaluate how AI models are performing across our products and content experiences

2. Design golden sets, taxonomies, and guidelines that enable consistent, repeatable content quality assessments

3. Build repeatable workflows for collecting, annotating, and analyzing AI outputs so evaluations can run efficiently as models evolve

4. Evaluate successive model releases through structured comparison, documenting what improved, what regressed, and what to prioritize next

5. Design evaluation frameworks that integrate qualitative and quantitative signals to measure dimensions like user trust, content depth, and topical relevance

6. Develop processes to track content quality and model performance over time and flag regressions

7. Synthesize evaluation results into structured error patterns and concrete recommendations that engineering and product teams can act on

8. Work cross-functionally with engineers, data scientists, product managers, and content strategists to align AI behaviors with real‑world user expectations

Minimum Qualifications:

Minimum Qualifications:

1. 5+ years of experience working collaboratively with product, engineering, design, and user research teams

2. 1+ years working with generative AI products, AI evaluation, prompt engineering, annotation, and/or content labeling and analysis

3. Experience designing and implementing evaluation frameworks, annotation guidelines, or quality rubrics for AI/ML systems

4. Demonstrated data analysis skills, with experience exploring data, identifying patterns, and producing actionable insights

5. Experience building new products or platform/ecosystem products

6. Critical thinking, experience leading data‑driven analyses to inform product or content decisions, and experience communicating to executive leadership

7. Proven track record of cross‑functional collaboration and delivering results in environments with evolving requirements and competing priorities

Preferred Qualifications:

Preferred Qualifications:

1. Background in content strategy, information quality, or trust and safety

2. Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)

3. Familiarity with AI evaluation methods such as human eval, model‑as‑judge, A/B testing, or red‑teaming

4. Experience building dashboards, scripts, or workflows that codify evaluation metrics

5. Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies

6. Experience with Python, SQL, or other tools for data analysis and evaluation automation

7. Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)

8. BA or BS in Computer Science, Data Science, Linguistics, or related field

Public Compensation:

$162,000/year to $227,000/year + bonus + equity + benefits

Industry: Internet

Equal Opportunity:

Meta is proud to be an Equal Employment Opportunity and Aff… ... ...

Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at [e-mail].