Tech Lead, Large-Scale ML Infra & Distributed Systems

TikTok USDS Joint Venture; Seattle (WA); On-site; USD 198,000 - 416,000; Full time

Job summary

TikTok USDS Joint Venture LLC is seeking an experienced software engineer to drive the technical roadmap for large-scale distributed ML training and inference systems powering TikTok's recommendations and search engines.

You will architect scalable training systems, optimize GPU utilization, and collaborate with ML research teams to deploy state-of-the-art models in production, while ensuring robust security and compliance across the compute stack.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related technical discipline.
• 5+ years in software engineering with Python and C++/Java and distributed systems experience.
• 3+ years building ML infrastructure at enterprise scale (GPU clusters, Kubernetes or Slurm).
• Deep familiarity with PyTorch and TensorFlow, CUDA, and GPU networking internals.
• Experience with production-grade LLM training/inference tools and profiling to remove bottlenecks.
• Strong system-level troubleshooting and debugging for I/O, compute, or network bottlenecks.

Responsibilities

• Drive the technical roadmap for large-scale distributed ML training and inferencing platforms.
• Architect and scale multi-node distributed training systems with 3D parallelism (Data, Tensor, Pipeline).
• Build low-latency, high-throughput model serving infrastructure for massive live traffic.
• Co-design next-generation Generative Recommendation components with Applied ML Research.
• Design automated fault-detection and asynchronous checkpointing for multi-thousand GPU clusters.
• Collaborate with researchers and data platform teams on high-throughput data processing and storage engines.
• Enforce strict encryption, access control, and tenant isolation across the compute stack.

Skills

Python

C++/Java

Distributed systems

Kubernetes

Slurm

CUDA

Education

Bachelor’s or Master’s degree in Computer Science/Engineering

Tools

Megatron

DeepSpeed

PyTorch

TensorFlow

Slurm

Job description

TikTok USDS Joint Venture LLC is seeking an experienced software engineer to drive the technical roadmap for large-scale distributed ML training and inference systems powering TikTok's recommendations and search engines.

You will architect scalable training systems, optimize GPU utilization, and collaborate with ML research teams to deploy state-of-the-art models in production, while ensuring robust security and compliance across the compute stack.