Data Quality & AI Operations Manager

extend; Riyadh; Hybrid; SAR 180,000 - 300,000; Full time

Job summary

Extend is seeking an experienced content and data operations lead to own the Extend platform. You will build a lean profiling process and use AI fine tuning to automate.

You will own evaluation loops and lead the content operations team across Riyadh and Shanghai time zones. You will define acceptance criteria, curate training data, and design human-in-the-loop systems while ensuring culture and language nuances are respected.

Qualifications

• Experience with Arabic/English labeling guidance and edge-case adjudication.
• Hands-on exposure to AI agent/LLM systems and eval design.
• Familiarity with training data, golden sets and model feedback loops.

Responsibilities

• Build and maintain golden sets and eval suites with acceptance criteria.
• Own data side of AI Admin iterations; curate training data and benchmarks.
• Design HITL mechanisms; feed corrections to prompts and data.
• Author labeling guidelines in Arabic and English; handle edge cases.
• Lead annotation operations: rater training, workflows, throughput, SLAs.
• Establish and track quality metrics: accuracy, automation, cost, turnaround.

Skills

Arabic language

AI data operations

Data labeling experience

Cross-border collaboration

Cultural context knowledge

Tools

LLM systems

AI eval design

Fine-tuning concepts

Job description

About Extend: Extend is a Saudi cultural intelligence and advertising company, operating since 2012, serving the Kingdom's leading brands and institutions across experiences, media, and publishing. We are building Extend CIP, the Cultural Intelligence Platform: one place to plan, manage, and measure all media activity, earned and paid, spanning media and influencer buying and a brand monitoring suite, with cultural intelligence and AI at its core.

About Extend: Extend is a Saudi cultural intelligence and advertising company, operating since 2012, serving the Kingdom's leading brands and institutions across experiences, media, and publishing. We are building Extend CIP, the Cultural Intelligence Platform: one place to plan, manage, and measure all media activity, earned and paid, spanning media and influencer buying and a brand monitoring suite, with cultural intelligence and AI at its core.

Role Summary

The role: Own content and data operations for the Extend platform. Build a leaner, more accurate profiling process and use AI fine tuning to progressively replace manual annotation with automation. You own the evaluation and data feedback loop and lead the content operations team. Success is measured by golden-set accuracy and profile automation rate.

What you'll do

• Build and maintain golden sets and eval suites; define acceptance criteria for each AI Admin iteration

• Own the data side of AI Admin iterations: curate training and evaluation data, run benchmarks, analyze failure modes, and partner with Shanghai engineering on fine tuning and deployment

• Design the human-in-the-loop (HITL) mechanism: define the boundary between human review and auto-approval, and feed human corrections back into prompts, eval sets and training data

• Author and maintain Arabic/English labeling guidelines; adjudicate edge cases requiring cultural and linguistic judgment

• Lead the content operations team with end-to-end ownership of annotation operations: rater training, workflows, throughput and quality SLAs

• Establish and track a quality metrics framework: accuracy, automation rate, inter-annotator agreement, cost and turnaround per profile

What we're looking for

• Fluent in Arabic, with strong knowledge of the Arab world's media, brands, publishers and cultural context

• Hands-on experience with AI agent / LLM systems: eval design, golden sets and model feedback loops; conversant in finetuning concepts (SFT, RLHF, prompt iteration) without needing to train models yourself

• 2+ years in AI data operations or applied AI quality, from a data labeling company (Scale AI, Surge, Appen, Sama, Invisible, Labelbox), an in-house annotation team at an AI company, or an AI product team

• Comfortable collaborating across Riyadh and Shanghai time zones

Nice to have:

• LLM-assisted labeling automation, taxonomy design, social or media data experience.