Model Data Operations Specialist, Evaluation & Judgement Center, Platform Responsibility

TikTok; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

TikTok is seeking a seasoned professional for the Platform Responsibility team in Singapore. You will drive AI safety initiatives, lead cross-functional collaboration across Product, Engineering and Operations, and optimize LLM-driven content risk assessment.

In this role you will refine strategy and prompts to enhance recall accuracy for risky content, while delivering scalable governance and data-driven improvements for global trust in the platform.

Qualifications

• Bachelor’s degree or above; 3+ years in model operations, AI/ML operations, or related domains.
• Proven ability to lead community policy development across Product, Engineering and Operations.
• Strong problem-solving skills with root-cause analysis and roadmap delivery.
• Familiarity with mainstream LLM products and model training/evaluation.

Responsibilities

• Undertake AI special projects for product R&D, annotate LLM training data, evaluate model iterations.
• Standardise definitions and create quantitative SOPs for Goldens Set across risk domains.
• Deliver ground truth through queue analysis, acceptance, customization and training.
• Establish an adjudication mechanism to align judgments on risk issues.

Skills

Model operations

Cross-functional leadership

Prompt engineering

AI/ML operations

Education

Bachelor's degree or above

Tools

ChatGPT

Gemini

Job description

About the Team

The Platform Responsibility team is bringing disruptive innovation to content safety through the deep application of AI. We are building next-generation infrastructure that leverages advanced LLMs to reshape risk detection and governance. By engineering high-quality data pipelines and integrating AI into core decision‑making, we drive exponential efficiency gains and precision. Working at the intersection of scalable engineering and technology, we empower global business growth while serving as the foundation of user trust.

About the Role

Under the spirit of TikTok Community Guidelines, the team leverages the capabilities of LLM and prompt engineering to deliver a large model Agent that can represent "absolute correctness", providing stable evaluation capabilities for TikTok content safety. This role will lead the team to refine strategy operations and prompt optimization to enhance the Agent's precise recall ability for risky content.

Responsibilities

• Undertake AI special projects for product R&D, perform tasks such as annotation of the LLM model training data sample set, evaluation of model iteration effectiveness, industry application research, and security compliance assurance. Cooperate closely with upstream product technology to jointly promote the achievement of business objectives.
• Standardise Definitions: Responsible for the definition and access criteria of the Golden Set across all risk domains (pornography, violence and terrorism, politics-related, abuse, etc.). Convert qualitative Policy documents into quantitative annotation SOPs executable by the Data Ops team.
• Ground Truth Delivery: Responsible for key aspects of queue delivery, including but not limited to demand analysis, acceptance, solution customization and training, quality assurance, delivery, etc.
• Operation of the Adjudication Mechanism: Establish an adjudication mechanism for internal cases to align consistent judgments on risk issues of different levels.
• Based on an understanding of LLM content safety process design and Community Guidelines, design rule routing logic suitable for LLM execution, and continuously optimize and iterate to improve model performance and resource utilization efficiency.
• Establish an AI content quality management system, be responsible for quality results, and continuously conduct work such as problem diagnosis, data analysis, and model optimization.

Minimum Qualifications

• Bachelor’s degree or above, with 3+ years of experience in model operations, AI/ML operations, or related domains.
• Demonstrated ability to lead community policy development and collaborate effectively across Product, Engineering, and Operations teams. Strong problem‑solving skills with the ability to identify root causes, prioritize initiatives, develop structured improvement roadmaps, drive execution, and measure impact through clear success metrics.
• Deep interest in internet products and strong awareness of AI industry trends. Familiarity with mainstream LLM products (e.g., ChatGPT, Gemini, etc.) and hands‑on experience in model training, evaluation, or prompt engineering.
• Highly data‑sensitive with strong analytical capabilities. Able to translate data insights into actionable strategies, synthesize complex findings, and proactively drive resolution of operational or model‑related issues.
• Demonstrated understanding of community safety frameworks and risk governance. Strong risk awareness and accountability, with the ability to proactively identify content, model, or process risks and design scalable mitigation solutions.

Preferred Qualifications

• Background in product management or technical experience in search systems, artificial intelligence, or large language models are strongly preferred.
• Self‑motivated and results‑driven, able to influence stakeholders in fast‑paced, ambiguous environments.

About TikTok

TikTok is the leading destination for short‑form mobile video. At TikTok, our mission is to inspire creativity and bring joy. TikTok's global headquarters are in Los Angeles and Singapore, and we also have offices in New York City, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo.

Why Join Us

Inspiring creativity is at the core of TikTok's mission. Our innovative product is built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and bring joy - a mission we work towards every day.

We strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. Every challenge is an opportunity to learn and innovate as one team. We're resilient and embrace challenges as they come. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At TikTok, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.