AI-Driven Search Quality Engineer

TikTok; Singapore; On-site; SGD 70,000 - 120,000; Full time

Job summary

TikTok is seeking a Software Test Engineer (AI) for TikTok Search in Singapore. You will work at the intersection of Search Engineering, Quality Engineering, and Generative AI to shape AI-native search quality.

Responsibilities include designing AI-powered testing, building evaluation pipelines, improving release quality gates, and collaborating with cross-functional teams to push AI-powered testing into production.

Qualifications

• Bachelor's degree or above in Computer Science, Software Engineering, Information Retrieval, Data Science, or a related technical discipline.
• Experience in software quality assurance or software testing, preferably for search, recommendation, ranking, content understanding, or other large-scale data- or algorithm-driven systems.
• Solid understanding of software testing methodologies, debugging, and quality assurance best practices across the software development lifecycle, including offline evaluation, online A/B testing, and production monitoring.
• Familiarity with distributed systems and search and recommendation infrastructure fundamentals (e.g. inverted indexes, vector indexes, retrieval, ranking, embeddings), plus basic knowledge of iOS and/or Android as search entry points.
• Hands-on experience in one or more testing domains, including automation testing, performance and stress testing, algorithm evaluation, or test platform and tool development.
• Strong interest in Generative AI and Large Language Models, with an understanding of common LLM capabilities, evaluation paradigms (e.g. LLM-as-a-Judge, RAG evaluation, and human evaluation), and real-world search applications.

Responsibilities

• You will work at the intersection of Search Engineering, Quality Engineering, and Generative AI, helping shape the future of AI-native Search Quality at TikTok.
• Design and deliver AI-powered testing and evaluation solutions for TikTok Search, covering query understanding, retrieval, ranking, generative search (LLM Answers and RAG), personalization, and multimodal and vertical search scenarios such as video, image, e-commerce, user, and creator search.
• Build LLM-based evaluation pipelines for search relevance, answer faithfulness, groundedness, freshness, coverage, diversity, and safety; construct and continuously maintain multilingual, multi-market query sets and golden datasets that reflect real user intent.
• Drive change impact analysis across the search stack—including index, retrieval, ranking models, and prompt and model changes—through AI-assisted diff analysis, offline replay, side-by-side (SBS) evaluation, and interleaving experiments; strengthen release quality gates and pre-launch verification capabilities.
• Partner closely with search engineering, algorithm, product, and content operations teams to identify quality and engineering productivity pain points across the search lifecycle, propose practical AI-powered solutions, and drive them into production.
• Improve engineering productivity for the search organization by building AI agents and tooling for automated test case generation, log-driven bug analysis, root-cause attribution, and self-healing regression suites.

Skills

Python

Go

Java

C++

Test automation

AI evaluation

Multilingual testing

Education

Bachelor's degree in Computer Science or related field

Tools

Job description

TikTok is seeking a Software Test Engineer (AI) for TikTok Search in Singapore. You will work at the intersection of Search Engineering, Quality Engineering, and Generative AI to shape AI-native search quality.

Responsibilities include designing AI-powered testing, building evaluation pipelines, improving release quality gates, and collaborating with cross-functional teams to push AI-powered testing into production.