AI Content Red Team Analyst - Trust and Safety

TikTok; San Jose (CA); On-site; USD 122,000 - 272,000; Full time

Job summary

TikTok's Trust & Safety GenAI & Emerging Products team seeks an experienced professional to conduct structured adversarial testing of AI models, features, and policies, to uncover vulnerabilities and emerging risks.

You will document findings with clear risk descriptions, reproduction steps, severity assessments, and mitigation recommendations, and collaborate with policy, product, engineering, and data science teams to improve safety before and after launch.

Qualifications

• Minimum 3 years in Trust & Safety, cybersecurity, risk/adversarial testing, or related fields.
• Experience with prompt testing, jailbreak analysis, LLM evaluation, or adversarial QA.
• Familiarity with AI safety risks (jailbreaks, hallucinations, bias, misuse patterns).
• Strong interest in GenAI safety and adversarial risk mitigation.
• Ability to investigate ambiguous problems and produce evidence-based conclusions.
• Ability to manage multiple priorities and collaborate across teams.

Responsibilities

• Conduct structured adversarial testing on AI models, features, and policies to identify vulnerabilities and emerging risks.
• Explore product behavior across contexts and user journeys to identify model failure modes not captured by standard evaluations.
• Investigate jailbreaks, evasions, prompt-based attacks, and other adversarial techniques relevant to content safety.
• Document findings with risk descriptions, reproduction steps, severity assessments, and mitigation recommendations.
• Partner with cross-functional stakeholders to ensure mitigation validation and root cause closure.
• Support development of testing playbooks, taxonomies, and internal knowledge bases.
• Stay updated on emerging adversarial trends and shifts in the external risk landscape.

Skills

Adversarial testing

GenAI safety

LLM evaluation

Jailbreak analysis

Risk assessment

Cross-functional collaboration

Ambiguous problem solving

Tools

Prompt testing tooling

Job description

Responsibilities

The Trust & Safety (T&S) GenAI & Emerging Product team's mission is to empower the development of GenAI models and applications. We do this by building a world-class safety, testing, and risk management system that ensures GenAI innovations are launched responsibly.

The AI Content Red Team sits within the T&S GenAI and Emerging Products pillar. The team is responsible for conducting unstructured adversarial testing of TikTok's generative AI products and models to uncover emerging risks, alongside our structured evaluations.

This team combines attacker-minded testing, risk discovery, and clear operational feedback loops to inform product decisions, policy development, mitigations, and longer-term evaluation strategy. We probe models and product experiences across modalities, use cases, and abuse patterns to identify failure modes, stress-test safeguards, and help teams improve safety before and after launch.

We work closely with Trust & Safety teams (policy, product, engineering, data science, operations), and business teams across global markets. Success in this team requires strong judgment, creativity, analytical rigor, and the ability to translate ambiguous findings into actionable recommendations.

Responsibilities:

• Conduct structured adversarial testing on AI models, features, and policies to identify vulnerabilities and emerging risks.
• Explore product behavior across contexts and user journeys, to identify model failure modes that may not be captured in standard evaluations.
• Investigate jailbreaks, evasions, prompt-based attacks, and other adversarial techniques relevant to content safety.
• Document findings clearly and consistently, including risk descriptions, reproduction steps, severity assessments, and mitigation recommendations.
• Partner with cross functional stakeholders (policy, product, business teams) to ensure mitigation validation and root cause closure.
• Support development of testing playbooks, taxonomies, and internal knowledge bases.
• Stay updated on emerging adversarial trends (e.g., deepfakes, multimodal manipulation, coordinated abuse), and shifts in the external risk landscape.

Qualifications

Minimum Qualification(s):

• Minimum 3 years of experience in Trust & Safety, cybersecurity, risk/adversarial testing, or related fields.
• Experience with prompt testing, jailbreak analysis, LLM evaluation, or adversarial QA.
• Familiarity with AI safety risks (jailbreaks, hallucinations, bias, misuse patterns).
• Strong interest in GenAI safety, and the ways AI systems can be compromised under adversarial conditions.
• Demonstrated ability to independently investigate ambiguous problems, identify non-obvious failure modes and abuse patterns, and produce clear, evidence-based conclusions.
• Ability to manage multiple priorities, and collaborate effectively with cross-functional teams.

Preferred Qualification(s):

• Experience working with agentic AI tools to scale your impact, including building/operating AI tools to make processes efficient and effective.

Trust & Safety

Content that this role interacts with includes images, video, and text related to every-day life, but it can also include (but is not limited to) bullying; hate speech; child safety; depictions of harm to self and others, and harm to animals. Hence, it is possible that this role will be exposed to harmful content on a daily basis.

TikTok recognises that keeping our platform safe for the TikTok communities is no ordinary job which can be both rewarding and psychologically demanding and emotionally taxing for some. This is why we are sharing the potential hazards, risks and implications in this unique line of work from the start, so our candidates are well informed before joining.

We are committed to the wellbeing of all our employees and promise to provide comprehensive and evidence-based programs, to promote and support physical and mental wellbeing throughout each employee's journey with us. We believe that wellbeing is a relationship and that everyone has a part to play, so we work in collaboration and consultation with our employees and across our functions in order to ensure a truly person-centred, innovative and integrated approach.

About TikTok

TikTok is the leading destination for short-form mobile video. At TikTok, our mission is to inspire creativity and bring joy. TikTok's global headquarters are in Los Angeles and Singapore, and we also have offices in New York City, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo.

Why Join Us

Inspiring creativity is at the core of TikTok's mission. Our innovative product is built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and bring joy - a mission we work towards every day.

We strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. Every challenge is an opportunity to learn and innovate as one team. We're resilient and embrace challenges as they come. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At TikTok, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

TikTok Accommodation

TikTok is committed to providing reasonable accommodations in our recruitment processes for candidates with disabilities, pregnancy, sincerely held religious beliefs or other reasons protected by applicable laws. If you need assistance or a reasonable accommodation, please reach out to us at https://tinyurl.com/RA-request

Job Information

【For Pay Transparency】Compensation Description (Annually)

The base salary range for this position in the selected city is $121600 - $272000 annually.

Compensation may vary outside of this range depending on a number of factors, including a candidate’s qualifications, skills, competencies and experience, and location. Base pay is one part of the Total Package that is provided to compensate and recognize employees for their work, and this role may be eligible for additional discretionary bonuses/incentives, and restricted stock units.

Benefits may vary depending on the nature of employment and the country work location. Employees have day one access to medical, dental, and vision insurance, a 401(k) savings plan with company match, paid parental leave, short-term and long-term disability coverage, life insurance, wellbeing benefits, among others. Employees also receive 10 paid holidays per year, 10 paid sick days per year and 17 days of Paid Personal Time (prorated upon hire with increasing accruals by tenure).

The Company reserves the right to modify or change these benefits programs at any time, with or without notice.

For Los Angeles County (unincorporated) Candidates:

Qualified applicants with arrest or conviction records will be considered for employment in accordance with all federal, state, and local laws including the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. Our company believes that criminal history may have a direct, adverse and negative relationship on the following job duties, potentially resulting in the withdrawal of the conditional offer of employment:

1. Interacting and occasionally having unsupervised contact with internal/external clients and/or colleagues;
2. Appropriately handling and managing confidential information including proprietary and trade secret information and access to information technology systems;
3. Exercising sound judgment.