AI Model Assurance Analyst

TikTok USDS Joint Venture; San Jose (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

TikTok USDS Joint Venture is seeking an AI Model Assurance Analyst to serve as the operational backbone of our GenAI oversight program. You will manage risk, change management, evidence pipelines, and independent validation to ensure releases meet safety commitments.

You will drive intake, evaluation, and decision packages, coordinating with legal, safety, and engineering teams while maintaining independence and adherence to SLAs.

Responsibilities

• Manage intake queue and progression of GenAI model and feature releases; validate submission completeness and drive SLAs.
• Classify submissions as material vs. non-material vs. emergency and route to evaluation teams
• Prepare approval-decision packages for the Content Assurance lead and log conditional approvals against post-launch monitoring commitments.
• Operationalize change-detection tooling and triage unnotified-change alerts
• Codify the performance envelope for each approval node against independent testing standards
• Watch alerts routing into the Content Assurance queue; run joint triage; log outcomes
• Support incident response; co-author remediation plans and post-incident reviews with teams
• Feed the audit evidence pipeline; track artifact readiness against evidence requests weekly
• Produce standing JV oversight reports on updates, incidents, SLA breaches, and remediation progress
• Drive weekly joint T&S / AISI triage, quarterly deep-dives, and annual program reviews
• Track program KPIs and surface trends to leadership
• Contribute to model assurance evaluation refreshes and the 'white glove' testing service for partner product teams
• Contribute to AI Model Assurance Dashboard, release pipeline automation, and notification loops across partners

Skills

GenAI oversight

Change management

Incident response

Job description

Responsibilities

About the Team JV AI & Systems Integrity is the independent function inside the TikTok USDS Joint Venture security organization that opines on whether generative AI features shipping to US users meet the JV's safety and content commitments. We operate the approval gates, testing programs, deviation adjudication, and audit-facing evidence pipeline for every JV-covered GenAI surface. We work in close partnership with legal, safety, and engineering teams.

About the Role The AI Model Assurance Analyst is the operational backbone of our GenAI oversight program. You will be the person who makes our control set (covering pre-launch approval, change management, change and deviation detection, inventory, independent validation, incident response, metrics reporting, and access segregation) run day-to-day: routing and completing intake and assessment tickets, driving evaluation dossiers to decisions, keeping the our risk and safety strategy registries accurate, monitoring performance, coordinating independent validation, and producing the monthly/quarterly oversight reports that go to JV leadership.

This is a hands-on operations role with meaningful policy and technical surface area. You will discuss and opine on release thresholds, develop strategies to catch unnotified changes, drive automation in our end-to-end process, and be the person a launch is waiting on when the evidence isn't good enough yet. We need someone who can hold this operational tempo without losing the thread on independence.

Responsibilities

• Manage the intake queue and progression of GenAI model and feature releases; validate submission completeness and be accountable for driving SLAs. Escalate risk of slippage proactively.
• Classify submissions as material vs. non-material vs. emergency and route them to appropriate evaluation teams
• Prepare approval-decision packages (approve / approve-with-conditions / reject / request more evidence) for the Content Assurance lead and log conditional approvals against post-launch monitoring commitments.
• Operationalize change-detection tooling (e.g., hash / configuration snapshots, deployment-event feed, canary-input probes) and triage unnotified-change alerts
• Codify the performance envelope for each approval node against our standards of independent testing
• Watch alerts routing into the Content Assurance queue in parallel with T&S; run joint triage; distinguish drift from change; drive deviation adjudication to closure and log outcomes
• Support CA incident response; co-author author remediation plans and post-incident reviews in partnership with our incident response team and compliance engineering teams.
• Feed the audit evidence pipeline; own artifact readiness against GRC's evidence request list and track completion percentage weekly.
• Produce the standing JV oversight reports on a regular cadence: adoptions and changes processed, incidents, SLA breaches, validation coverage, open remediation, and other details.
• Drive the weekly joint T&S / AISI triage, the quarterly rotating deep-dive, and the annual program review and threshold recalibration.
• Track program-level KPIs and surface trends and blockers to leadership.
• Contribute to model assurance evaluation refreshes (risk-based eval, neutrality test sets, safety testing framework and SOP) and the "white glove" testing service for partner product teams
• Contribute to the AI Model Assurance Dashboard, model release pipeline automation, and notification loops across all cross-functional partners for an integrated source of truth on independent testing status

Qualifications

Minimum Qualifications