Content Engineer

Sarafu; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Sarafu in Singapore is seeking a Content Engineer focused on evaluating generative media quality. You will design rubrics, run evaluation rounds, and translate subjective judgments into measurable signals that guide model training and launches.

You will work with product, research, and modeling teams, applying rigorous quality standards to high-volume video outputs and developing reusable evaluation frameworks for scalable impact.

Qualifications

• 6+ years in video production, content strategy or media evaluation.
• Hands-on video production including shooting/editing/post or motion work.
• Experience creating or contributing to evaluation rubrics or quality guidelines.
• Experience producing or evaluating short-form video and emerging formats.
• Ability to document judgment logic into reusable frameworks.
• Experience leveraging quantitative insights to inform content decisions.
• Cross-functional collaboration with product, engineering, research, analytics

Responsibilities

• Build and maintain evaluation rubrics for image/video quality, defining scales and guidelines.
• Run human evaluation rounds, write clear, actionable reports.
• Create prompt sets and golden sets to stress-test model capabilities.
• Coordinate with researchers to translate eval findings into training signals.
• Help transition human judgment into automated measurements and track trends.

Skills

Video production

Editorial standards

Content evaluation

Cross-functional collaboration

Quantitative insights

Rubric documentation

Education

Bachelor's degree

Tools

Annotation platforms

Eval tooling

Job description

Product Content Engineering is a horizontal function supporting initiatives across Instagram and Meta AI, and is seeking a content engineer to join the team. We partner closely with product, research and modeling teams to solve problems with content-driven solutions, set standards of quality, and define what “good” looks like for generative media experiences. This position is focused on model training for image and video generation capabilities. You will own the human evaluation layer that tells our models what quality means, designing the rubrics, running the eval rounds, and turning subjective creative judgment into reproducible, measurable signals that directly shape model behavior and launch decisions. This position requires hands‑on video production experience and the craft vocabulary to describe media quality precisely. The work involves applying rubrics consistently across high volumes of generated clips, distinguishing prompt‑following failures from model‑quality failures, documenting observations clearly, and writing and refining the rater guidelines themselves.

Responsibilities

• Build and maintain evaluation rubrics for image and video generation quality, including dimension definitions, rating scales and annotator guidelines.
• Execute human evaluation rounds on model output, side‑by‑side, absolute scoring and prompt‑adherence checks, and produce clear, actionable writeups.
• Construct and curate prompt sets and golden sets that stress‑test specific model capabilities and failure modes.
• Calibrate with fellow raters and vendor teams; help drive inter‑rater reliability.
• Partner with research and modeling teams so eval findings translate into training and post‑training signal.
• Support the transition of human judgment into automated measurement and help validate automated scores against human ground truth.
• Track quality trends across model versions and flag regressions early.
• Deliver high‑quality evaluation work on model and launch timelines in a fast‑paced, dynamic environment.

Minimum Qualifications

• 6+ years of experience in video production, content strategy, editorial standards, media evaluation or relevant fields
• Hands‑on video production experience, including shooting, editing, post and/or motion work, with the craft vocabulary to describe quality failures precisely
• Experience creating or substantially contributing to evaluation rubrics or quality guidelines for video or image content
• Experience producing or evaluating short‑form video and emerging video formats
• Experience applying editorial judgment, plus the ability to document judgment logic into reusable frameworks that scale across teams
• Experience leveraging quantitative insights to inform content and quality decisions
• Experience collaborating with cross‑functional teams (product, engineering, design, research, analytics)
• Experience working independently and navigating ambiguity in fast‑paced environments
• Bachelor's degree in a directly related field, or equivalent practical experience

Preferred Qualifications

• Experience evaluating generative media output (text-to-video, image-to-video, video-to-video, text-to-image)
• Background in film, television, OTT, commercial or VFX production
• Familiarity with generative media tooling and prompt engineering for media generation
• Experience with annotation platforms or eval tooling
• Experience with product launches in ambiguous or zero-to-one environments
• Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
• Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
• Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies