Technical AI Policy Researcher, Frontier Risk - Trust and Safety

TikTok; San Francisco (CA); On-site; USD 108,000 - 209,000; Full time

Benefits offered by this job

Health insurance

401(k) with match

Paid parental leave

Wellbeing benefits

Job summary

TikTok is seeking a Technical AI Policy Researcher to advance responsible AI across our frontier models, partnering with safety researchers, engineers, and product teams. You will design policies, evaluate risks, and help operationalize safeguards to ensure safe, fair, and trustworthy AI at scale.

You will translate risk models into concrete requirements, drive end-to-end policy workflows from pre-launch to post-launch, and collaborate with external experts to strengthen policy defensibility and

Qualifications

• 5 years in Trust & Safety, AI Safety Research, AI Ethics, technical AI Governance, or equivalent experience.
• Advanced degree in Computer Science, Human-Computer Interaction, Engineering, Data Science or quantitative Social Sciences.
• Direct experience in frontier risk research, AI evaluations, red-teaming, or AI governance work.
• Strong technical understanding of LLM, multimodel, or genmedia model behavior, model failure modes, and safety risks.
• Demonstrated experience working with external experts and stakeholders, including civil society, government, and academia.
• Demonstrated success working in a fast-paced technology company or research organization conducting AI impact, risk assessments or algorithmic audits, and/or data science or product development related experience.
• Ability to advocate for safety amongst a wide variety of business stakeholders including Product Policy, Engineering, Public Policy, Legal, Communications, and Data Science.

Responsibilities

• Design and maintain multimodal GenAI policies across safety-relevant domains, including frontier risk in agentic modalities, loss of control or deceptive misalignment.
• Translate risk and harm models into clear behavioral specifications, evaluation criteria, grading guidance, and system-level safeguards.
• Define practical boundaries between beneficial uses of AI and assistance that could materially enable harm, exploitation, misuse, or unsafe outcomes.
• Build policy artifacts that support model training, evaluation, and deployment. Partner with safety researchers, engineers, product teams, and other stakeholders to operationalize policy into scalable model behavior and measurable safeguards.
• Design end-to-end policy to pre-launch evaluation to post-launch monitoring workflows across safety-relevant domains, including golden set construction, labeling guidance, calibration, adjudication, and eval coverage analysis, to ensure policies can be reliably measured and improved.
• Use red-teaming results, deployment data, model failures, over-refusals, under-refusals, and ambiguous edge cases to improve policy and evaluation quality over time.
• Identify emerging capability areas where frontier AI systems could create new safety, fairness or bias challenges or lower barriers to harm.
• Monitor post-launch model activity to identify gaps in our policy framework to capture unsafe model behaviour.
• Champion research to strengthen the defensibility and operability of policy positions, including working with Outreach and Partnerships to incorporate external expert input into relevant policy positions.
• Combine longer-horizon safety research with hands-on launch and deployment work.
• Contribute to risk reports, policy documentation, launch reviews, and AI governance reviews on the company's approach to building AI responsibly.
• Support regulatory teams as a subject matter expert on AI compliance related initiatives.

Skills

Trust & Safety

Policy Research

AI Safety

Governance

Stakeholder Engagement

Education

Advanced degree

Job description

Responsibilities

The Trust & Safety (T&S) Responsible AI Policy team's mission is to ensure the development of GenAI models and applications are safe, fair and trustworthy. We do this by defining, measuring and mitigating safety and fairness AI model risks through policy frameworks, model risk assessments, and upstream policy solutions.

The T&S Responsible AI Policy team sits within the T&S GenAI and Emerging Products pillar. We work closely with Trust & Safety teams (product policy, product, engineering, data science, operations, red teaming), business and model teams, and cross-functional stakeholders (comms, legal, public policy) across global markets. Success in this team requires strong policy acumen, judgment, creativity, analytical rigour, and the ability to translate Generative AI risk to different stakeholders effectively.

As a Technical AI Policy Researcher on the T&S Responsible AI Policy team, you will champion the responsible development and deployment of our frontier AI models across multiple businesses with a specialty on technical research for frontier risks. You will accelerate technical policy research, incubate new research efforts, and drive end-to-end policy to evaluate workflows for your domain areas.

• Design and maintain multimodal GenAI policies across safety-relevant domains, including frontier risk in agentic modalities, loss of control or deceptive misalignment.
• Translate risk and harm models into clear behavioral specifications, evaluation criteria, grading guidance, and system-level safeguards.
• Define practical boundaries between beneficial uses of AI and assistance that could materially enable harm, exploitation, misuse, or unsafe outcomes.
• Build policy artifacts that support model training, evaluation, and deployment. Partner with safety researchers, engineers, product teams, and other stakeholders to operationalize policy into scalable model behavior and measurable safeguards.
• Design end-to-end policy to pre-launch evaluation to post-launch monitoring workflows across safety-relevant domains, including golden set construction, labeling guidance, calibration, adjudication, and eval coverage analysis, to ensure policies can be reliably measured and improved.
• Use red-teaming results, deployment data, model failures, over-refusals, under-refusals, and ambiguous edge cases to improve policy and evaluation quality over time.
• Identify emerging capability areas where frontier AI systems could create new safety, fairness or bias challenges or lower barriers to harm.
• Monitor post-launch model activity to identify gaps in our policy framework to capture unsafe model behaviour.
• Champion research to strengthen the defensibility and operability of policy positions, including working with Outreach and Partnerships to incorporate external expert input into relevant policy positions.
• Combine longer-horizon safety research with hands-on launch and deployment work.
• Contribute to risk reports, policy documentation, launch reviews, and AI governance reviews on the company's approach to building AI responsibly.
• Support regulatory teams as a subject matter expert on AI compliance related initiatives.

Qualifications

Minimum Qualifications:

• 5 years in Trust & Safety, AI Safety Research, AI Ethics, technical AI Governance, or equivalent experience.
• Advanced degree in Computer Science, Human-Computer Interaction, Engineering, Data Science or quantitative Social Sciences
• Direct experience in frontier risk research, AI evaluations, red-teaming, or AI governance work.
• Strong technical understanding of LLM, multimodel, or genmedia model behavior, model failure modes, and safety risks.
• Demonstrated experience working with external experts and stakeholders, including civil society, government, and academia.
• Demonstrated success working in a fast-paced technology company or research organization conducting AI impact, risk assessments or algorithmic audits, and/or data science or product development related experience.
• Ability to advocate for safety amongst a wide variety of business stakeholders including Product Policy, Engineering, Public Policy, Legal, Communications, and Data Science.

Preferred Qualifications:

• Technical knowledge in high efficiency on device architectures, multimodal understanding, V&V of AI systems, or RAG is beneficial but not required.
• Experience working with governments, frontier AI companies, or AI Safety organizations.
• Experience working in non-western cultures, with a particular emphasis on the global south.
• Understanding of Trust & Safety positioning in the entertainment media technology sector, with comfort learning internal tools and product workflows.

Trust & Safety

Content that this role interacts with includes images, video, and text related to every-day life, but it can also include (but is not limited to) bullying; hate speech; child safety; depictions of harm to self and others, and harm to animals. Hence, it is possible that this role will be exposed to harmful content on a daily basis.

TikTok recognises that keeping our platform safe for the TikTok communities is no ordinary job which can be both rewarding and psychologically demanding and emotionally taxing for some. This is why we are sharing the potential hazards, risks and implications in this unique line of work from the start, so our candidates are well informed before joining.

We are committed to the wellbeing of all our employees and promise to provide comprehensive and evidence-based programs, to promote and support physical and mental wellbeing throughout each employee's journey with us. We believe that wellbeing is a relationship and that everyone has a part to play, so we work in collaboration and consultation with our employees and across our functions in order to ensure a truly person-centred, innovative and integrated approach.

About TikTok

TikTok is the leading destination for short-form mobile video. At TikTok, our mission is to inspire creativity and bring joy. TikTok's global headquarters are in Los Angeles and Singapore, and we also have offices in New York City, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo.

Inspiring creativity is at the core of TikTok's mission. Our innovative product is built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and bring joy - a mission we work towards every day.

We strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. Every challenge is an opportunity to learn and innovate as one team. We're resilient and embrace challenges as they come. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At TikTok, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach.

We are passionate about this and hope you are too.

TikTok Accommodation

TikTok is committed to providing reasonable accommodations in our recruitment processes for candidates with disabilities, pregnancy, sincerely held religious beliefs or other reasons protected by applicable laws. If you need assistance or a reasonable accommodation, please reach out to us at https://tinyurl.com/RA-request

Job Information

【For Pay Transparency】Compensation Description (Annually)

The base salary range for this position in the selected city is $108000 - $208800 annually.

Compensation may vary outside of this range depending on a number of factors, including a candidate’s qualifications, skills, competencies and experience, and location. Base pay is one part of the Total Package that is provided to compensate and recognize employees for their work, and this role may be eligible for additional discretionary bonuses/incentives, and restricted stock units.

• Employees have day one access to medical, dental, and vision insurance, a 401(k) savings plan with company match, paid parental leave, short-term and long-term disability coverage, life insurance, wellbeing benefits, among others.
• Employees also receive 10 paid holidays per year, 10 paid sick days per year and 17 days of Paid Personal Time (prorated upon hire with increasing accruals by tenure).

The Company reserves the right to modify or change these benefits programs at any time, with or without notice.

For Los Angeles County (unincorporated) Candidates:

1. Interacting and occasionally having unsupervised contact with internal/external clients and/or colleagues;

2. Appropriately handling and managing confidential information including proprietary and trade secret information and access to information technology systems; and

3. Exercising sound judgment.