Content Engineer, Meta Superintelligence Labs

Meta

New York (NY)

On-site

USD 131,000 - 190,000

Full time

6 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Bonus
Equity
Benefits

Job summary

Meta Superintelligence Labs is seeking a Content Engineer to improve AI product capabilities and agentic experiences. You will design evaluation materials, apply quality standards, and convert user feedback into measurable improvements across models and product surfaces.

You will collaborate with research, engineering, product, and design teams to define quality criteria, build rubrics, and run human evaluation programs, while refining system prompts and agent workflows for scalable impact.

Qualifications

  • 4+ years in digital content strategy, user experience, technical writing, journalism, production, or related fields.
  • Experience with human evaluation workflows, rubric development, golden set construction, and calibration.
  • Ability to translate qualitative feedback into structured quality criteria, failure categories, and measurable benchmarks.

Responsibilities

  • Define and apply quality criteria for AI product capabilities; build guidelines, golden sets, and benchmarks.
  • Design and run human evaluation workstreams; calibrate results and validate against automated signals.
  • Run structured testing programs; design test plans and triage results for product and engineering teams.
  • Craft and refine system prompts and agentic workflows in partnership with technical teams.
  • Partner with engineering, research, product and design on quality standards and priorities.

Skills

Digital content strategy
User experience
Editorial judgment
Technical writing
Cross-functional collaboration

Education

Bachelor's degree in a related field

Tools

LLM tooling
Annotation platforms
Prototyping environments

Job description

Content Engineering is a horizontal function within Meta Superintelligence Labs (MSL) that shapes AI product experiences through prompt engineering, evaluations, and quality frameworks. Sitting at the intersection of user experience and large language model behavior, content engineers partner closely with research science, engineering, product, and design teams to build and ship AI experiences from lab to production, across modalities and surfaces.MSL is seeking a content engineer to improve the quality of AI product capabilities and agentic experiences. In this role, you'll independently create and improve model evaluations and training data content-quality within assigned product/model behavior areas to measure and improve the model behavior on a variety of different tasks. You'll define and apply quality standards, build evaluation materials, and turn qualitative feedback from internal and external users into measurable findings that inform model and product improvements. You'll combine editorial judgment, analytical skills, and cross-functional collaboration to help create experiences that entertain, inform, and delight.Content Engineer, Meta Superintelligence Labs Responsibilities:Quality Definition & Frameworks: Define and apply quality criteria for assigned AI product capabilities. Build and maintain guidelines, golden response sets, and benchmarks across features and surfaces. Develop failure-mode taxonomies that turn user-facing quality issues into structured, prioritized findings for technical partnersEvaluation & Measurement: Design and run human evaluation workstreams, including rubric development, annotator guidance, calibration, and analysis of results. Partner with research science to support automated evaluation and validate automated quality signals against human ratings. Track quality trends and regressions across model versions and product capabilitiesTesting & Iteration: Run structured internal testing programs, design test plans targeting specific failure modes, triage results, and deliver prioritized recommendations to product and engineering within development timelines. Craft and refine system prompts and agentic behavior in partnership with technical teams to support product goals and model outcomesCross-Functional Partnership: Partner with engineering, research science, product, policy, and design teams on quality standards and priorities for assigned product areas. Communicate evaluation findings clearly, align partners on next steps, and independently deliver workstreams through influence and collaborationAgentic Workflows: Use AI-native tools to build scalable, repeatable processes for evaluation design, data analysis, system prompting, and reporting. Experiment with emerging tools, validate their outputs, and document workflows that improve quality and reduce manual workMinimum Qualifications:Bachelor's degree in a directly related field, or equivalent practical experience4+ years of experience in digital content strategy, user experience, technical writing, journalism, production, or related fieldsExperience with human evaluation workflows, including rubric or guideline development, golden set construction, and calibrationExperience translating qualitative content or user-experience feedback into structured quality criteria, failure categories, and measurable benchmarksExperience designing and running structured testing programs, analyzing results, and communicating prioritized findings to technical partnersExperience making editorial and content-quality decisions, communicating complex concepts to cross-functional partners, and independently delivering work in environments with evolving requirements and competing prioritiesPreferred Qualifications:Experience in prompt engineering and agentic workflowsDemonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologiesExperience with AI-native tooling, such as LLM-based development tools, annotation platforms, or prototyping environments, and integrating those tools into workflows to improve efficiency or qualityDemonstrated ongoing AI skill development, such as prompt/context engineering or agent orchestration, and staying current with emerging AI technologiesExperience applying responsible AI practices, including bias mitigation and quality and accuracy reviewsExperience with LLM-as-judge development, including building automated quality signals aligned with human judgment and validating that alignment over timeExperience working with product teams or programs from roadmapping through deliveryExperience with generative AI training or model evaluation, including creating or curating human-annotated or synthetic datasetsExperience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)About Meta:Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.Meta is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment.Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at accommodations-ext@meta.com.$131,000/year to $190,000/year + bonus + equity + benefitsIndividual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, Meta offers benefits. Learn more about benefits at Meta.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Product Content Engineer
Product Content Engineer

Instagram • San Francisco (CA)

On-site
USD 162,000 - 227,000
Product Content Engineer
Product Content Engineer

Instagram • United States

On-site
USD 162,000 - 227,000
Lead, Product Content Engineering
Lead, Product Content Engineering

Meta • Menlo Park (CA)

On-site
USD 193,000 - 269,000
Product Content Engineer - Creative Evals
Product Content Engineer - Creative Evals

Meta • San Francisco (CA)

On-site
USD 131,000 - 190,000
Product Content Engineer
Product Content Engineer

Vamp • Menlo Park (CA)

On-site
USD 131,000 - 190,000
Bonus
Equity
Benefits
Product Content Engineer
Product Content Engineer

Meta • Utah

On-site
USD 162,000 - 227,000
Communications Manager, AI Products
Communications Manager, AI Products

Meta • San Francisco (CA)

On-site
USD 146,000 - 206,000
Bonus
Equity
Benefits
Product Content Engineer
Product Content Engineer

Meta • Nashville (TN)

On-site
USD 162,000 - 227,000
Product Content Engineer
Product Content Engineer

Meta • Salem (OR)

On-site
USD 162,000 - 227,000
Software Engineer, Machine Learning - Ads Infra and Ranking
Software Engineer, Machine Learning - Ads Infra and Ranking

Meta • Sunnyvale (CA)

On-site
USD 184,000 - 257,000
Bonus
Equity
Benefits