Content Engineer, Meta Superintelligence Labs

Meta; Menlo Park (CA); Hybrid; USD 140,000 - 210,000; Full time

Job summary

Meta is seeking a Content Engineer to shape AI product experiences by defining and enforcing quality benchmarks across prompts, evaluations, and outputs. You will build scalable evaluation pipelines, translate user feedback into concrete metrics, and collaborate with research, engineering, product, and policy teams to improve model alignment and user satisfaction.

You will design test plans, manage annotators and golden sets, and drive practical improvements in production AI experiences,

Qualifications

• Bachelor's degree or equivalent experience with 5+ years in digital content strategy, user experience, technical writing, journalism, production, or related fields.
• Experience designing and running human evaluation pipelines at scale — including annotator management, rubric design, golden set construction, and calibration.
• Experience translating ambiguous feedback into structured, objective, fixable categories.
• Experience defining and operationalizing subjective quality dimensions into measurable benchmarks.
• Experience running structured software testing/QA programs — designing test plans, triaging results, and delivering actionable analysis to engineering teams fast enough to matter.
• Experience making editorial and content quality decisions in a fast-paced environment, communicating complex technical concepts to cross-functional partners, and leading through influence across teams without direct reporting lines.
• Experience with AI-native tooling and a bias toward using them to move faster.
• Experience with LLM-as-judge development — building automated quality signals aligned with human judgment, and validating alignment over time.
• Experience working with product teams or programs from roadmapping through delivery.
• Experience working in prompt engineering and agentic workflows.
• Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact.
• Experience adhering to responsible, ethical AI practices (risk assessment, bias mitigation, quality and accuracy reviews).
• Demonstrated ongoing AI skill development (prompt/context engineering, agent orchestration).

Responsibilities

• Own the quality bar for AI-powered product experiences and translate qualitative insights into measurable benchmarks.
• Build and operate evaluation infrastructure (human and automated) to measure model performance at product scale.
• Serve as a cross-functional bridge between research science, engineering, product, and policy to prioritize fixes.
• Define what great outputs look like and drive feedback loops to improve models and workflows.

Skills

Human evaluation pipelines

Ambiguity translation to structure

Quality benchmarks

Software testing/QA

Editorial and content quality

AI-native tooling

LLM-as-judge development

Cross-functional collaboration

Prompt engineering

Agent orchestration

AI ethics practices

AI skill development

Education

Bachelor's degree or equivalent experience

Tools

Annotation platforms

Prototyping environments

LLM-based development tools

Job description

Content Engineering is a horizontal function in the Muse Product Post Training org within Meta Superintelligence Labs (MSL) that shapes AI product experiences by aligning models to be helpful in new product experience through productizing prompt engineering, frontier evaluations, and quality frameworks. Sitting at the intersection of user experience and large-language model behavior, content engineers partner closely with research science, engineering, product and design teams to build and ship AI experiences from lab to production, across modalities and surfaces.

Great Content Engineers have the aesthetic taste to notice what makes a model output great, the technical prompting expertise to align models to consistently provide great responses, experience at building auto-gradeable frontier evals for new capabilities, and the experience delivering high scale datasets via human annotators and synthetic data. This role goes beyond shaping how the model communicates — it owns the quality bar. This person defines what "great" looks like across product capabilities, builds the evaluation infrastructure (human and automated) to measure it, and runs the feedback loops that turn qualitative insight into model improvement. They operate as a cross-functional bridge between research science, engineering, product, and policy, translating user-facing quality problems into structured priorities and actionable fixes. They serve as a point of contact for internal stakeholders across AI on priority product and model initiatives, and aim to build experiences that entertain, inform, and delight.

• Bachelor's degree or equivalent experience with 5+ years of experience in digital content strategy, user experience, technical writing, journalism, production, or related fields
• Experience designing and running human evaluation pipelines at scale — including annotator management, rubric design, golden set construction, and calibration
• Experience translating ambiguous "it doesn't feel right" feedback into structured, objective, fixable categories
• Experience defining and operationalizing subjective quality dimensions into measurable benchmarks
• Experience running structured software testing/qa programs — designing test plans, triaging results, and delivering actionable analysis to engineering teams fast enough to matter in the current dev cycle
• Experience making editorial and content quality decisions in a fast-paced environment, communicating complex technical concepts to cross-functional partners, and leading through influence across teams without direct reporting lines
• Experience with AI-native tooling (LLM-based development tools, annotation platforms, prototyping environments) and a bias toward using them to move faster
• Experience with LLM-as-judge development — building automated quality signals aligned with human judgment, and validating that alignment over time
• Experience working with product teams or programs (or other equivalent fields) from roadmapping through delivery
• Experience working in prompt engineering and agentic workflows
• Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
• Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
• Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies