Content Engineer, Meta Superintelligence Labs

Meta; New York (NY); On-site; USD 162,000 - 227,000; Full time

Benefits offered by this job

Bonus

Equity

Benefits

Job summary

Meta's Content Engineering team within Meta Superintelligence Labs seeks a Content Engineer to define quality standards for AI product capabilities, build evaluation infra, and drive alignment between research, engineering, product, and policy.

You will design rubrics, manage annotator teams, run large-scale evals, and lead fast iteration across prompts and system behavior to deliver reliable, user-friendly AI experiences.

Qualifications

• Bachelor's degree or equivalent; 5+ years in digital content strategy, UX, or related fields.
• Experience designing and running human evaluation pipelines at scale.
• Experience translating ambiguous feedback into structured categories.
• Experience defining quality benchmarks for product capabilities.
• Experience running structured software testing/QA with fast feedback to engineering.

Responsibilities

• Define quality standards for AI product capabilities and build guidelines.
• Own the full human evaluation pipeline and analyze results.
• Design and run large-scale evals to track quality over time.
• Lead testing programs and triage results for rapid iteration.
• Coordinate across product, engineering, research, and design on quality.

Skills

Prompt engineering

Quality frameworks

Human evaluation design

Cross-functional leadership

Agentic workflows

Education

Bachelor's degree or equivalent

Tools

LLM-based development tools

Annotation platforms

Prototyping environments

Job description

Content Engineering is a horizontal function in the Muse Product Post Training org within Meta Superintelligence Labs (MSL) that shapes AI product experiences by aligning models to be helpful in new product experience through productizing prompt engineering, frontier evaluations, and quality frameworks. Sitting at the intersection of user experience and large-language model behavior, content engineers partner closely with research science, engineering, product and design teams to build and ship AI experiences from lab to production, across modalities and surfaces.Great Content Engineers have the aesthetic taste to notice what makes a model output great, the technical prompting expertise to align models to consistently provide great responses, experience at building auto-gradeable frontier evals for new capabilities, and the experience delivering high scale datasets via human annotators and synthetic data. This role goes beyond shaping how the model communicates — it owns the quality bar. This person defines what "great" looks like across product capabilities, builds the evaluation infrastructure (human and automated) to measure it, and runs the feedback loops that turn qualitative insight into model improvement. They operate as a cross-functional bridge between research science, engineering, product, and policy, translating user-facing quality problems into structured priorities and actionable fixes. They serve as a point of contact for internal stakeholders across AI on priority product and model initiatives, and aim to build experiences that entertain, inform, and delight.
Content Engineer, Meta Superintelligence Labs Responsibilities:

• Quality Definition & Frameworks: Define what "great" looks like for AI product capabilities — build guidelines, golden response sets, and frontier evals that set the quality bar across features and surfaces. Develop failure mode taxonomies that give engineering teams a structured, prioritized view of what's breaking and why.
• Evaluation & Measurement: Own the full human evaluation pipeline — design rubrics, guide contractor annotator teams, build calibration processes, and deliver data analysis on results. Partner with Research Science to build and validate auto-judges aligned with human raters
• define the methodology for measuring alignment drift. Construct and run large-scale evals to track quality metrics across product capabilities over time.
• Testing & Iteration: Lead structured dogfooding and testing programs — design test plans targeting specific failure modes, run testing rounds, triage and categorize results, and deliver prioritized summaries to product and engineering. Operate at speed to unblock fast iteration — turn around quality assessments quickly enough to inform the current dev cycle, not the next one. Identify emerging quality issues before they reach external users. Craft and tune system prompts and agentic behavior to support product vision and model outcomes.
• Cross-Functional Leadership: Serve as a key member of the cross-functional team across the product capabilities you work on— aligning engineering, research science, product, policy, and design on quality standards and priorities. Lead through influence and collaboration across teams
• make quality legible and actionable for technical partners.
• Agentic workflows: Leverage AI-native tools to replace manual workflows with scalable, repeatable processes across evaluation design, data analysis, visual development and cross-functional comms. Continuously evaluate and adopt emerging tools that allow the team to operate at speed. This is a role where experimentation is key, and a willingness to experiment, learn new tools, and build solutions hands‑on is valued.
  

Minimum Qualifications:

• Bachelor's degree or equivalent experience with 5+ years of experience in digital content strategy, user experience, technical writing, journalism, production, or related fields
• Experience designing and running human evaluation pipelines at scale — including annotator management, rubric design, golden set construction, and calibration
• Experience translating ambiguous "it doesn't feel right" feedback into structured, objective, fixable categories
• Experience defining and operationalizing subjective quality dimensions into measurable benchmarks
• Experience running structured software testing/qa programs — designing test plans, triaging results, and delivering actionable analysis to engineering teams fast enough to matter in the current dev cycle
• Experience making editorial and content quality decisions in a fast‑paced environment, communicating complex technical concepts to cross‑functional partners, and leading through influence across teams without direct reporting lines
  

Preferred Qualifications:

• Experience with AI‑native tooling (LLM‑based development tools, annotation platforms, prototyping environments) and a bias toward using them to move faster
• Experience with LLM‑as‑judge development — building automated quality signals aligned with human judgment, and validating that alignment over time
• Experience working with product teams or programs (or other equivalent fields) from roadmapping through delivery
• Experience working in prompt engineering and agentic workflows
• Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
• Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
• Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
  

About Meta:
Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.
Meta is proud to be an Equal Employment Opportunity and Aff…? Meta participates in the E‑Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment.
Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at [e-mail].
$162,000/year to $227,000/year + bonus + equity + benefits
Individual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, Meta offers benefits. Learn more about benefits at Meta.