Seattle ML Engineer: Self-Evolving Conversational AI Agent

TikTok; Seattle (WA); On-site; USD 122,000 - 243,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

401(k) with company match

Paid parental leave

Disability coverage

Life insurance

Wellbeing benefits

10 holidays per year

10 paid sick days per year

17 days of Paid Personal Time

Job summary

TikTok is seeking a candidate to build a self-evolving unified conversational agent for its global e-Commerce customer service in Seattle. You will design the agent runtime, safety gates, memory management, and evaluation pipelines to transform conversations into training signals.

The role requires a Bachelor’s in a technical field, strong Python and one of C++, Go, Rust, or Java, and hands-on ML/NLP experience with LLMs or agents.

Qualifications

• Completing or recently completing a Bachelor’s degree in Computer Science, Electrical Engineering, Mathematics, Statistics, or a related discipline
• Strong Python plus one of: C++, Go, Rust, Java
• Solid fundamentals in ML/DL/NLP with hands-on experience using LLMs or agents (coursework, research, internship, competition, open-source, or a serious side project)
• Basic statistical literacy: confidence intervals, p-values interpretation, and understanding the difference between “a number went up” and real system improvement
• Ability to read a research paper or engineering blog and translate it into working code

Responsibilities

• Agent runtime (harness / agent loop): orchestrate skills, tools, and context; implement loop control and intervention; apply progressive disclosure; add behavior-level guardrails
• Production safety layer: create pre-flight budgets and timeout truncation; implement serve-time gates; deliver shadow and swap-in answers; define safe fallback paths
• Context & memory for long multi-turn agents: build agentic memory via structured note-taking; perform context compaction and summarization; support context editing and observation masking; use retrieve-then-load for just-in-time retrieval
• Post-training and the data flywheel: apply SFT, DPO, and RL so rules move into model weights (reducing prompt length); distill to smaller serving models; convert served conversations into training, evaluation, and retrieval signals
• Tools, skills, and MCP: design tools as APIs and connectors; enable skill and tool search for large inventories; govern a skill library (handle description conflicts, trigger evaluations, maintain cross-skill mis-fire matrices); load skills on demand instead of placing every definition into context
• Evaluation you can launch with: use LLM-as-judge with human-agreement calibration; apply statistical rigor (paired comparison, confidence intervals, repeated sampling, pass^k); run held-out and time-rolling evaluation splits with overfitting alarms; support cascaded scoring and cross-family judge panels to scale evaluation cost
• Self-evolving loop: implement case mining to derive root cause; generate constrained candidate sets; replay-verify against frozen regression sets; run canary checks and feed results into the flywheel
• Auditable systems: build candidate registry with exact runtime read-back, change lineage, and an archive of rejected candidates for sampling in later rounds
• Online experimentation and causal readout: run shadow, canary, and A/B experiments; use non-inferiority gates and traffic-split health; define metrics that hold up under scrutiny; apply off-policy counterfactual evaluation when live A/B is not possible
• Location & employment details: Location: Seattle, WA (onsite) with Regular employment and Start: 2027

Skills

Python

C++

Go

Rust

Java

ML/DL/NLP

Education

Bachelor's degree in Computer Science, Electrical Engineering, Mathematics, Statistics

Tools

vLLM

TensorRT-LLM

MoE

KV

A/B testing

Job description

TikTok is seeking a candidate to build a self-evolving unified conversational agent for its global e-Commerce customer service in Seattle. You will design the agent runtime, safety gates, memory management, and evaluation pipelines to transform conversations into training signals.

The role requires a Bachelor’s in a technical field, strong Python and one of C++, Go, Rust, or Java, and hands-on ML/NLP experience with LLMs or agents.