Senior Data Scientist: AI Risk & Continuous Audit

TikTok; San Jose (CA); On-site; USD 137,000 - 293,000; Full time

Job summary

TikTok is seeking a data science/AI auditing specialist for its Internal Audit team in the US. You will power continuous auditing by building data products, evaluating LLMs, and delivering actionable insights across multiple verticals.

The role focuses on model lifecycle audits, data warehousing, ETL pipelines, and automated analytics to strengthen controls and risk management. Strong technical and communication skills are essential.

Qualifications

• Bachelor's degree in a quantitative field and 5+ years of data science/AI experience with LLMs.
• Experience designing, deploying, and monitoring large-scale ML models and audits.
• Strong SQL and Python skills with ML libraries (Transformers, TensorFlow, PyTorch).
• Ability to define and assess model performance metrics and bias/fairness quantification.
• Experience auditing AI/ML models in corporate, regulatory, or advisory contexts.

Responsibilities

• Audit model lifecycles from training to deployment and monitoring for quality, performance, and risk.
• Identify model vulnerabilities including bias, fairness violations, and security risks; propose remediation.
• Define and measure metrics such as accuracy, precision/recall, F1, calibration, and robustness.
• Communicate findings and risk assessments clearly to technical and non-technical stakeholders.
• Partner with auditors and data owners to provide data support and testing strategies.
• Develop data warehouses and ETL pipelines to support audit engagements and dashboards.

Skills

LLM auditing frameworks

Model evaluation

SQL

Python

Transformer LLMs

Bias / fairness assessment

Data analytics

Communication with stakeholders

ETL pipelines

Education

Bachelor's degree in quantitative discipline

PhD (preferred)

Tools

Hugging Face Transformers

TensorFlow

PyTorch

Airflow / ML pipeline orchestration

AWS

GCP

Azure

Snowflake

Job description

TikTok is seeking a data science/AI auditing specialist for its Internal Audit team in the US. You will power continuous auditing by building data products, evaluating LLMs, and delivering actionable insights across multiple verticals.

The role focuses on model lifecycle audits, data warehousing, ETL pipelines, and automated analytics to strengthen controls and risk management. Strong technical and communication skills are essential.