Product Content Engineer

Instagram

United States

On-site

USD 162,000 - 227,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Meta's Product Content Engineering team seeks a Content Engineer to partner with our agent tooling group to define model quality standards and target behaviors for Meta’s internal AI assistant. You will design evaluation frameworks, build rubrics, and pipelines to hold AI outputs to a high standard, and you’ll work with engineering, product, research, and data science to improve performance.

You’ll own the establishment of golden sets, guidelines, and workflows for analyzing AI outputs, tracking

Qualifications

  • Bachelor's degree or equivalent practical experience in Computer Science, Data Science, Linguistics, or related field.
  • 5+ years collaborating with product, engineering, design, and user research teams.
  • 2+ years working with generative AI, evaluation, prompting, annotation, or labeling.
  • Experience designing evaluation frameworks, guidelines, or quality rubrics for agentic systems.
  • Data analysis skills; ability to identify patterns and produce actionable insights.
  • Ability to convey findings to executive leadership and influence product decisions.
  • Proven track record of cross-functional collaboration in evolving environments.

Responsibilities

  • Define quality standards for model actions, task completion, and permission requests.
  • Design golden sets, taxonomies, and guidelines integrating qualitative and quantitative signals.
  • Build repeatable workflows for collecting, annotating, and analyzing AI outputs.
  • Evaluate model releases with structured comparisons and document changes.
  • Develop processes to track model quality and flag regressions over time.
  • Synthesize evaluation results into actionable recommendations for engineering and product teams.
  • Collaborate with engineering, data science, and product teams to align AI behaviors with user expectations.
  • Identify and mitigate operational risks before delivery.

Skills

Cross-functional collaboration
AI evaluation
Prompt engineering
Data analysis
Executive communication
Decision-making

Education

Bachelor's degree or equivalent practical experience in CS/Data Science/Linguistics

Tools

Python
SQL

Job description

Product Content Engineering is a horizontal function supporting initiatives across Meta. We partner closely with product and technical teams to solve problems by providing model-centered solutions, setting standards of quality, and building the frameworks that ensure AI-powered experiences actually work for people. We’re looking for a Content Engineer who will partner with Meta’s agent tooling team to define model quality standards and target behaviors for Meta’s internal AI assistant – an automated tool driving employee productivity across internal systems, data, and workflows. You’ll work at the intersection of evaluation: building the frameworks, rubrics, and pipelines that hold AI outputs to a high standard. You’ll assess model behavior and hardness, identify where it falls short, and work cross-functionally with engineering, product, research, and data science teams to make it better. If you’re energized by building better AI experiences through rigorous evaluation, thrive in ambiguity, and bring strong editorial and analytical judgment.

Product Content Engineer Responsibilities
  • Define quality standards – how well the model is at taking actions, completing tasks, changing states, asking for permission when needed, etc. and use them to systematically evaluate how AI models are performing for Meta’s internal AI assistant.
  • Design golden sets, taxonomies, and guidelines that integrate qualitative and quantitative signals.
  • Build repeatable workflows for collecting, annotating, and analyzing AI outputs so evaluations can run efficiently as models evolve.
  • Evaluate successive model releases through structured comparison, documenting what improved, what regressed, and what to prioritize next.
  • Develop processes to track model quality and performance over time and flag regressions.
  • Synthesize evaluation results into structured error patterns and concrete recommendations that engineering and product teams can act on.
  • Work cross-functionally with engineering, data science, and product teams to align AI behaviors with real-world user expectations.
  • Identify and mitigate operational risks — staffing gaps, timeline conflicts, quality regressions — before they impact delivery.
Minimum Qualifications
  • Bachelor's degree in Computer Science, Data Science, Linguistics, or a related field, or equivalent practical experience
  • 5+ years of experience working collaboratively with product, engineering, design, and user research teams
  • 2+ years working with generative AI products, AI evaluation, prompt engineering, annotation, and/or labeling and analysis
  • Experience designing and implementing evaluation frameworks, annotation guidelines, or quality rubrics for agentic systems
  • Demonstrated data analysis skills, with experience exploring data, identifying patterns, and producing actionable insights
  • Experience applying critical thinking to lead data-driven analyses that inform product or user experience decisions, and communicating findings to executive leadership
  • Proven track record of cross-functional collaboration and delivering results in environments with evolving requirements and competing priorities
Preferred Qualifications
  • Experience establishing ground truth for text-based agentic workflows (e.g., multi-step reasoning, tool use, end-to-end task completion), including the annotation guidelines and quality rubrics that make evaluations repeatable
  • Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
  • Familiarity with AI evaluation methods such as human eval, model-as-judge, A/B testing, or red-teaming
  • Experience building dashboards, scripts, or workflows that codify evaluation metrics
  • Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
  • Experience with Python, SQL, or other tools for data analysis and evaluation automation
  • Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
About Meta

Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.

Meta is proud to be an Equal Employment Opportunity and Aff…

Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at accommodations-ext@meta.com.

$162,000/year to $227,000/year + bonus + equity + benefits

Individual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, Meta offers benefits. Learn more about benefits at Meta.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Product Content Engineer
Product Content Engineer

Instagram • San Francisco (CA)

On-site
USD 162,000 - 227,000
Product Content Engineer
Product Content Engineer

Meta • Utah

On-site
USD 162,000 - 227,000
Product Content Engineer
Product Content Engineer

Meta • Nashville (TN)

On-site
USD 162,000 - 227,000
Product Content Engineer
Product Content Engineer

Meta • Salem (OR)

On-site
USD 162,000 - 227,000
Product Content Engineer
Product Content Engineer

Vamp • Menlo Park (CA)

On-site
USD 131,000 - 190,000
Bonus
Equity
Benefits
Lead, Product Content Engineering
Lead, Product Content Engineering

Vamp • Los Angeles (CA)

On-site
USD 193,000 - 269,000
Content Engineer, Meta Superintelligence Labs
Content Engineer, Meta Superintelligence Labs

Meta • New York (NY)

On-site
USD 131,000 - 190,000
Bonus
Equity
Benefits
Member of Product Staff
Member of Product Staff

Vamp • Seattle (WA)

On-site
USD 205,000 - 277,000
Creator and Content Benchmarking Lead, Product Content Engineer
Creator and Content Benchmarking Lead, Product Content Engineer

Instagram • New York (NY)

On-site
USD 193,000 - 269,000
Member of Product Staff
Member of Product Staff

Instagram • San Francisco (CA)

On-site
USD 205,000 - 277,000