Content Quality and Evaluation Specialist - AI Data Service Operations

TikTok; Kuala Lumpur; On-site; MYR 120,000 - 180,000; Full time

Job summary

TikTok is seeking an Eco Content Quality and Evaluation Specialist to gatekeep AI training data, define quality standards, and ensure data trustworthiness across multilingual, multimodal datasets. You will shape judgment rules and collaborate with cross-functional teams to align production with quality acceptance feedback.

The role focuses on safeguarding data quality for AI training and evaluation, influencing governance strategy and model performance across global platforms.

Qualifications

• Bachelor's degree or above in data, statistics, computer science, law, linguistics, or social sciences preferred.
• Internship, academic project, campus org, or full-time experience in data quality, QA, content moderation, compliance, or AI data/annotation is a strong plus.
• Excellent English proficiency (listening, speaking, reading, writing) with accurate comprehension of English video and text content.
• Proficient in Excel and common data tools; hands-on experience with mainstream LLM products and a basic understanding of AI capabilities.
• Strong logical thinking with structured problem-decomposition, abstraction, and articulation; detail-oriented with a view on data quality.
• Ability to interpret and apply complex guidelines in writing-focused workflows; internship, research, and project experience count.

Responsibilities

• Design and iterate data quality standards, judgment rules, and acceptance criteria for LLM training and evaluation.
• Review data outputs against quality standards and implement corrections for consistency and accuracy.
• Tackle complex edge cases, drive discussions, and convert conclusions into reusable rules and assets.
• Track quality metrics, root causes, and drive improvements in production processes.
• Collaborate cross-functionally to align on quality standards across production, product, algorithm, and policy teams.

Skills

English proficiency

Excel and data tools

Structured problem solving

Policy/guidelines interpretation

Data quality/QA experience

LLM data / annotation experience

Education

Bachelor's degree or above in data, statistics, CS, law, linguistics, or social sciences

Tools

ChatGPT

Claude

Gemini

AI data tools

Job description

Responsibilities

About the Team

Generative AI and large language models are reshaping the core capabilities of global content platforms — and high-quality training and evaluation data is one of the key factors that determines a model's ceiling.

We are the AI Data Service & Operations team behind TikTok's international products, responsible for producing multilingual, multimodal data assets across both safety and non-safety domains. The standards we set and the data we deliver power two critical fronts: TikTok's global content ecosystem governance strategy on one side, and the training and evaluation pipelines for AI / large language models on the other. In short, we produce the fuel that helps models truly understand content, communities, and users around the world.

As an Eco Content Quality and Evaluation Specialist, you'll sit at the intersection of model performance and ecosystem governance. You'll act as the gatekeeper for AI training and evaluation data — defining quality standards, unifying judgment criteria, and safeguarding data trustworthiness and consistency, so that every data point holds up under the scrutiny of both models and the business.

Key Responsibilities

1. Help design and iterate on data quality standards, judgment rules, and acceptance criteria for LLM training and evaluation scenarios;
2. Review and adjudicate data outputs against quality standards, identify issues, and close the loop on corrections to ensure accuracy and consistency;
3. Dig into complex, ambiguous, and edge cases; drive discussion and turn conclusions into reusable judgment rules and knowledge assets;
4. Track quality metrics, root-cause issues, and drive improvements in production processes and execution;
5. Partner cross-functionally with production, product, algorithm, and policy teams to align on quality standards and connect the "standard to production to acceptance to feedback" loop.

Qualifications

Minimum Qualifications

• Bachelor's degree or above, majors in data, statistics, computer science, law, linguistics, or social sciences preferred;
• Internship, academic project, campus organization, or full-time experience in data quality, quality assurance, content moderation, compliance, content operations, editorial review, or AI data / annotation is a strong plus;
• Excellent English proficiency (listening, speaking, reading, writing) with accurate comprehension of English video and text content; IELTS 7.5 / TOEFL 105 or equivalent
• Proficient in Excel and common data tools; hands-on experience with mainstream LLM products (ChatGPT, Claude, Gemini, etc.) and a basic understanding of AI capability boundaries preferred;
• Strong logical thinking with structured problem-decomposition, abstraction, and articulation skills; detail-oriented, with your own point of view on "what makes good data."
• Demonstrated ability to interpret and apply complex guidelines or policies in writing-focused workflows, and some experience evaluating qualitative content or using data to improve processes. Internship, research, and project experience all count.

Preferred Qualifications

• Additional experience with content policy, operational guideline development, or enforcement workflows.
• A self-starter mindset, solution-oriented thinking, and the ability to manage multiple priorities in a fast-paced, collaborative environment.
• Familiarity with machine-executable logic, labeling frameworks, or test-set workflows.
• Experience collaborating with Policy, Product, Governance, Engineering, or Training teams through internships, projects, research, or full-time roles.
• Knowledge of scenario coverage, positive/negative balance, and dataset validation.

About TikTok

TikTok is the leading destination for short-form mobile video. At TikTok, our mission is to inspire creativity and bring joy. TikTok's global headquarters are in Los Angeles and Singapore, and we also have offices in New York City, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo.

Why Join Us

Inspiring creativity is at the core of TikTok's mission. Our innovative product is built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and bring joy – a mission we work towards every day.

We strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. Every challenge is an opportunity to learn and innovate as one team. We're resilient and embrace challenges as they come. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At TikTok, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.