Software Test Engineer (AI), TikTok Search

TikTok; Singapore; Vor Ort; SGD 70.000 - 120.000; Vollzeit

Zusammenfassung

TikTok is seeking a Software Test Engineer (AI) for TikTok Search in Singapore. You will work at the intersection of Search Engineering, Quality Engineering, and Generative AI to shape AI-native search quality.

Responsibilities include designing AI-powered testing, building evaluation pipelines, improving release quality gates, and collaborating with cross-functional teams to push AI-powered testing into production.

Qualifikationen

• Bachelor's degree or above in Computer Science, Software Engineering, Information Retrieval, Data Science, or a related technical discipline.
• Experience in software quality assurance or software testing, preferably for search, recommendation, ranking, content understanding, or other large-scale data- or algorithm-driven systems.
• Solid understanding of software testing methodologies, debugging, and quality assurance best practices across the software development lifecycle, including offline evaluation, online A/B testing, and production monitoring.
• Familiarity with distributed systems and search and recommendation infrastructure fundamentals (e.g. inverted indexes, vector indexes, retrieval, ranking, embeddings), plus basic knowledge of iOS and/or Android as search entry points.
• Hands-on experience in one or more testing domains, including automation testing, performance and stress testing, algorithm evaluation, or test platform and tool development.
• Strong interest in Generative AI and Large Language Models, with an understanding of common LLM capabilities, evaluation paradigms (e.g. LLM-as-a-Judge, RAG evaluation, and human evaluation), and real-world search applications.

Aufgaben

• You will work at the intersection of Search Engineering, Quality Engineering, and Generative AI, helping shape the future of AI-native Search Quality at TikTok.
• Design and deliver AI-powered testing and evaluation solutions for TikTok Search, covering query understanding, retrieval, ranking, generative search (LLM Answers and RAG), personalization, and multimodal and vertical search scenarios such as video, image, e-commerce, user, and creator search.
• Build LLM-based evaluation pipelines for search relevance, answer faithfulness, groundedness, freshness, coverage, diversity, and safety; construct and continuously maintain multilingual, multi-market query sets and golden datasets that reflect real user intent.
• Drive change impact analysis across the search stack—including index, retrieval, ranking models, and prompt and model changes—through AI-assisted diff analysis, offline replay, side-by-side (SBS) evaluation, and interleaving experiments; strengthen release quality gates and pre-launch verification capabilities.
• Partner closely with search engineering, algorithm, product, and content operations teams to identify quality and engineering productivity pain points across the search lifecycle, propose practical AI-powered solutions, and drive them into production.
• Improve engineering productivity for the search organization by building AI agents and tooling for automated test case generation, log-driven bug analysis, root-cause attribution, and self-healing regression suites.

Kenntnisse

Python

Go

Java

C++

Test automation

AI evaluation

Multilingual testing

Ausbildung

Bachelor's degree in Computer Science or related field

Tools

Jobbeschreibung

Software Test Engineer (AI), TikTok Search

Location:

Employment Type:

Regular

Job Code:

A43305

Share this listing:

Responsibilities

You will work at the intersection of Search Engineering, Quality Engineering, and Generative AI, helping shape the future of AI-native Search Quality at TikTok.

• Design and deliver AI-powered testing and evaluation solutions for TikTok Search, covering query understanding, retrieval, ranking, generative search (LLM Answers and RAG), personalization, and multimodal and vertical search scenarios such as video, image, e-commerce, user, and creator search.
• Build LLM-based evaluation pipelines for search relevance, answer faithfulness, groundedness, freshness, coverage, diversity, and safety; construct and continuously maintain multilingual, multi-market query sets and golden datasets that reflect real user intent.
• Drive change impact analysis across the search stack—including index, retrieval, ranking models, and prompt and model changes—through AI-assisted diff analysis, offline replay, side-by-side (SBS) evaluation, and interleaving experiments; strengthen release quality gates and pre-launch verification capabilities.
• Partner closely with search engineering, algorithm, product, and content operations teams to identify quality and engineering productivity pain points across the search lifecycle, propose practical AI-powered solutions, and drive them into production.
• Improve engineering productivity for the search organization by building AI agents and tooling for automated test case generation, log-driven bug analysis, root-cause attribution, and self-healing regression suites.

Qualifications

Minimum Qualifications

• Bachelor's degree or above in Computer Science, Software Engineering, Information Retrieval, Data Science, or a related technical discipline.
• Experience in software quality assurance or software testing, preferably for search, recommendation, ranking, content understanding, or other large-scale data- or algorithm-driven systems.
• Solid understanding of software testing methodologies, debugging, and quality assurance best practices across the software development lifecycle, including offline evaluation, online A/B testing, and production monitoring.
• Familiarity with distributed systems and search and recommendation infrastructure fundamentals (e.g. inverted indexes, vector indexes, retrieval, ranking, embeddings), plus basic knowledge of iOS and/or Android as search entry points.
• Hands-on experience in one or more testing domains, including automation testing, performance and stress testing, algorithm evaluation, or test platform and tool development.
• Strong interest in Generative AI and Large Language Models, with an understanding of common LLM capabilities, evaluation paradigms (e.g. LLM-as-a-Judge, RAG evaluation, and human evaluation), and real-world search applications.

Preferred Qualifications

• Proficiency in one or more programming languages such as Python, Go, Java, or C++.
• Hands-on experience evaluating or testing search, recommendation, ranking, or LLM and RAG systems, including relevance labeling, SBS evaluation, NDCG, MRR, and Recall analysis, faithfulness and groundedness measurement, or hallucination detection.
• Experience developing test automation frameworks, evaluation platforms, query set management tools, or engineering productivity tools for search and algorithm teams.
• Hands-on experience with Generative AI–related projects, including LLM application development, AI agent evaluation, model evaluation, AI-assisted testing, prompt engineering, fine-tuning, or building AI testing frameworks.
• Experience leveraging AI tools (e.g. LLMs, AI coding assistants, or AI agents) to improve testing efficiency, automate test case generation, bug analysis, or search quality workflows.
• Experience with multilingual and cross-market quality work, content safety testing, or vertical search domains such as e-commerce, video, or creator search.

Job Information

About TikTok

TikTok is the leading destination for short-form mobile video. At TikTok, our mission is to inspire creativity and bring joy. TikTok's global headquarters are in Los Angeles and Singapore, and we also have offices in New York City, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo.

Why Join Us

Inspiring creativity is at the core of TikTok's mission. Our innovative product is built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and bring joy – a mission we work towards every day.

We strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. Every challenge is an opportunity to learn and innovate as one team. We're resilient and embrace challenges as they come. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At TikTok, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Ready for a career

at TikTok?

Discover a career that energizes and excites you every day.