AI Model Assurance Analyst

TikTok USDS Joint Venture; New York (NY); On-site; USD 90,000 - 130,000; Full time

Job summary

TikTok USDS Joint Venture is seeking an AI Model Assurance Analyst to operate the GenAI oversight program within the security organization. You will manage intake and assessment tickets, drive evaluation dossiers to decisions, and maintain risk and safety registries, while producing monthly and quarterly oversight reports for JV leadership.

This hands-on role covers pre-launch approval, change management, incident response, and independent validation, with a focus on automation and end-to-end

Job description

Responsibilities

About the Team JV AI & Systems Integrity is the independent function inside the TikTok USDS Joint Venture security organization that opines on whether generative AI features shipping to US users meet the JV's safety and content commitments. We operate the approval gates, testing programs, deviation adjudication, and audit-facing evidence pipeline for every JV-covered GenAI surface. We work in close partnership with legal, safety, and engineering teams.

About the Role The AI Model Assurance Analyst is the operational backbone of our GenAI oversight program. You will be the person who makes our control set (covering pre-launch approval, change management, change and deviation detection, inventory, independent validation, incident response, metrics reporting, and access segregation) run day-to-day: routing and completing intake and assessment tickets, driving evaluation dossiers to decisions, keeping the our risk and safety strategy registries accurate, monitoring performance, coordinating independent validation, and producing the monthly/quarterly oversight reports that go to JV leadership.

This is a hands-on operations role with meaningful policy and technical surface area. You will discuss and opine on release thresholds, develop strategies to catch unnotified changes, drive automation in our end-to-end process, and be the person a launch is waiting on when the evidence isn't good enough yet. We need someone who can hold this operational tempo without losing the thread on independence.

Responsibilities

• Manage the intake queue and progression of GenAI model and feature releases; validate submission completeness and be accountable for driving SLAs. Escalate risk of slippage proactively.
• Classify submissions as material vs. non-material vs. emergency and route them to appropriate evaluation teams
• Prepare approval-decision packages (approve / approve-with-conditions / reject / request more evidence) for the Content Assurance lead and log conditional approvals against post-launch monitoring commitments.
• Operationalize change-detection tooling (e.g., hash / configuration snapshots, deployment-event feed, canary-input probes) and triage unnotified-change alerts
• Codify the performance envelope for each approval node against our standards of independent testing
• Watch alerts routing into the Content Assurance queue in parallel with T&S; run joint triage; distinguish drift from change; drive deviation adjudication to closure and log outcomes
• Support CA incident response; co-author author remediation plans and post-incident reviews in partnership with our incident response team and compliance engineering teams.
• Feed the audit evidence pipeline; own artifact readiness against GRC's evidence request list and track completion percentage weekly.
• Produce the standing JV oversight reports on a regular cadence: adoptions and changes processed, incidents, SLA breaches, validation coverage, open remediation, and other details.
• Drive the weekly joint T&S / AISI triage, the quarterly rotating deep-dive, and the annual program review and threshold recalibration.
• Track program-level KPIs and surface trends and blockers to leadership.
• Contribute to model assurance evaluation refreshes (risk-based eval, neutrality test sets, safety testing framework and SOP) and the "white glove" testing service for partner product teams
• Contribute to the AI Model Assurance Dashboard, model release pipeline automation, and notification loops across all cross-functional partners for an integrated source of truth on independent testing status

Qualifications

Minimum Qualifications