AI Assurance Engineer - MLE Evaluation & Safety

TikTok USDS Joint Venture; San Jose (CA); On-site; USD 187,000 - 438,000; Full time

Job summary

TikTok USDS Joint Venture is seeking an AI Assurance Engineer to design rigorous evaluation methods and production-grade capabilities for measuring recommender and GenAI behavior. You will develop benchmarks, define protocols, implement safety and drift metrics, and investigate regressions through adversarial testing.

Role focuses on evaluation engineering and technical assurance, not model training or policy setting.

Qualifications

• Education: Bachelor degree or higher in Computer Science, Machine Learning, Statistics, or related field, or equivalent experience.
• Machine Learning Evaluation: Experience evaluating ML systems with defined metrics, datasets, baselines, and reproducible protocols.
• Software Engineering: Production-quality evaluation or data-processing software in Python, Go, or C++.

Responsibilities

• Define reproducible evaluation protocols and benchmark suites for recommender and GenAI evaluations.
• Develop benchmark datasets, sampling plans, and safety/robustness metrics.
• Implement evaluation code, replay workflows, and telemetry signals on shared infrastructure.
• Diagnose results and distinguish model regressions from data or infra effects.
• Integrate benchmarks and metrics with CI/CD and evaluation platforms.

Skills

Machine Learning evaluation

Software engineering

Statistical rigor

Communication

Education

Bachelor's degree or higher in CS/ML/Statistics or related field

Tools

Python

Go

C++

Job description

TikTok USDS Joint Venture is seeking an AI Assurance Engineer to design rigorous evaluation methods and production-grade capabilities for measuring recommender and GenAI behavior. You will develop benchmarks, define protocols, implement safety and drift metrics, and investigate regressions through adversarial testing.

Role focuses on evaluation engineering and technical assurance, not model training or policy setting.