Product Content Engineer

Meta

Geldern

On-site

EUR 147,000 - 206,000

Full time

40 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Equity
Bonus

Job summary

Meta looks for a Content Engineer to partner with the agent tooling team to set model quality standards and target behaviors for Meta’s internal AI assistant — an automated tool enhancing employee productivity across internal systems, data and workflows.

You will build evaluation frameworks, rubrics and pipelines, assess model behavior, and work with engineering, product, research and data science to improve AI outputs and reliability.

Qualifications

  • 5+ years of experience collaborating with product, engineering, design and user research teams.
  • 2+ years working with generative AI products, evaluation, prompt engineering, annotation or labeling.
  • Experience designing evaluation frameworks, guidelines or quality rubrics for agentic systems.
  • Proven data analysis skills with ability to identify patterns and generate insights.
  • Ability to communicate findings to executive leadership and drive data-driven decisions.
  • Strong cross-functional collaboration and delivery in evolving priority environments.

Responsibilities

  • Define quality standards for model actions, task completion, state changes and permission requests.
  • Design golden sets, taxonomies, and guidelines integrating qualitative and quantitative signals.
  • Build repeatable workflows for collecting, annotating, and analyzing AI outputs.
  • Evaluate releases through structured comparison and document improvements and regressions.
  • Develop processes to track model quality and flag regressions over time.
  • Synthesize results into actionable recommendations for engineering and product teams.
  • Collaborate across engineering, data science and product to align AI behaviors with user expectations.
  • Identify and mitigate operational risks before delivery.

Skills

Cross-functional collaboration
Editorial judgment
AI evaluation
Model quality standards
Data analysis
Communication to leadership

Education

Bachelor's degree

Tools

Python
SQL

Job description

Summary:

Product Content Engineering is a horizontal function supporting initiatives across Meta. We partner closely with product and technical teams to solve problems by providing model-centered solutions, setting standards of quality, and building the frameworks that ensure AI-powered experiences actually work for people.We're looking for a Content Engineer who will partner with Meta’s agent tooling team to define model quality standards and target behaviors for Meta’s internal AI assistant - an automated tool driving employee productivity across internal systems, data, and workflows. You'll work at the intersection of evaluation: building the frameworks, rubrics, and pipelines that hold AI outputs to a high standard. You'll assess model behavior and hardness, identify where it falls short, and work cross-functionally with engineering, product, research, and data science teams to make it better.If you're energized by building better AI experiences through rigorous evaluation, thrive in ambiguity, and bring strong editorial and analytical judgment, we encourage you to apply.

Required Skills:

Product Content Engineer Responsibilities:

  1. Define quality standards - how well the model is at taking actions, completing tasks, changing states, asking for permission when needed, etc. and use them to systematically evaluate how AI models are performing for Meta’s internal AI assistant.
  2. Design golden sets, taxonomies, and guidelines that integrate qualitative and quantitative signals.
  3. Build repeatable workflows for collecting, annotating, and analyzing AI outputs so evaluations can run efficiently as models evolve.
  4. Evaluate successive model releases through structured comparison, documenting what improved, what regressed, and what to prioritize next.
  5. Develop processes to track model quality and performance over time and flag regressions.
  6. Synthesize evaluation results into structured error patterns and concrete recommendations that engineering and product teams can act on.
  7. Work cross-functionally with engineering, data science, and product teams to align AI behaviors with real-world user expectations.
  8. Identify and mitigate operational risks - staffing gaps, timeline conflicts, quality regressions - before they impact delivery.
Minimum Qualifications:
  1. 5+ years of experience working collaboratively with product, engineering, design, and user research teams
  2. 2+ years working with generative AI products, AI evaluation, prompt engineering, annotation, and/or labeling and analysis
  3. Experience designing and implementing evaluation frameworks, annotation guidelines, or quality rubrics for agentic systems
  4. Demonstrated data analysis skills, with experience exploring data, identifying patterns, and producing actionable insights
  5. Experience applying critical thinking to lead data-driven analyses that inform product or user experience decisions, and communicating findings to executive leadership
  6. Proven track record of cross-functional collaboration and delivering results in environments with evolving requirements and competing priorities
Preferred Qualifications:
  1. Experience establishing ground truth for text-based agentic workflows (e.g., multi-step reasoning, tool use, end-to-end task completion), including the annotation guidelines and quality rubrics that make evaluations repeatable
  2. Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
  3. Familiarity with AI evaluation methods such as human eval, model-as-judge, A/B testing, or red-teaming
  4. Experience building dashboards, scripts, or workflows that codify evaluation metrics
  5. Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
  6. Experience with Python, SQL, or other tools for data analysis and evaluation automation
  7. Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
Public Compensation:

$162,000/year to $227,000/year + bonus + equity + benefits

Industry:

Internet

Equal Opportunity:

Meta is proud to be an Equal Employment Opportunity and Aff…? Wait. (the paragraph omitted)

Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at accommodations-ext@meta.com.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Software Engineer, Multimedia & Multimodal AI
Software Engineer, Multimedia & Multimodal AI

Meta • Geldern

On-site
EUR 167,000 - 233,000
Member of Product Staff
Member of Product Staff

Meta • Geldern

On-site
EUR 186,000 - 251,000
Backend Engineer, Central Growth
Backend Engineer, Central Growth

Meta • Geldern

On-site
EUR 167,000 - 233,000
Privacy Data Analyst, Evidencing and Monitoring
Privacy Data Analyst, Evidencing and Monitoring

Meta • Geldern

On-site
EUR 139,000 - 189,000
Analytics Program Manager
Analytics Program Manager

Meta • Geldern

On-site
EUR 131,000 - 186,000
Product Security Engineer - Team Lead
Product Security Engineer - Team Lead

Meta • Geldern

On-site
EUR 198,000 - 273,000
Bonus
Equity
Benefits
Staff AI Product Engineer
Staff AI Product Engineer

Royper Royco LLC • Germany

Remote
EUR 181,000 - 272,000
Stock options
Paid time off 18 days
Public holidays 11
+4
SEO & Digital Marketing Expert
SEO & Digital Marketing Expert

SME Careers • Germany

Remote
EUR 55,000 - 83,000
Hardware Systems Architect, AI Hardware Products
Hardware Systems Architect, AI Hardware Products

Meta • Geldern

On-site
EUR 188,000 - 262,000
Social Media Expert
Social Media Expert

SME Careers • Germany

Remote
EUR 55,000 - 110,000