Tech Lead - Large-Scale ML Infra & GPU Systems

TikTok USDS Joint Venture; San Jose (CA); On-site; USD 187,000 - 438,000; Full time

Job summary

TikTok USDS Joint Venture LLC is seeking experienced ML engineers to lead and evolve the large-scale real-time training and inference platforms powering its TikTok recommendations. You will drive architecture for multi-node GPU clusters, ensure low-latency serving under heavy traffic, and collaborate with ML researchers to push generative recommendation capabilities in production.

The role requires deep experience with PyTorch/TensorFlow, CUDA, and distributed systems, plus leadership to mentor

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or related technical discipline.
• 5+ years of professional software engineering experience with Python, C++/Java and distributed systems.
• 3+ years building ML infrastructure at enterprise scale (GPU clusters, Kubernetes, or Slurm).
• Deep knowledge of PyTorch or TensorFlow and low-level GPU memory management.

Responsibilities

• Lead technical roadmap for large-scale real-time ML training and inference platforms powering TikTok recommendations and search.
• Architect multi-node distributed training systems with 3D parallelism strategies.
• Build low-latency, high-throughput model serving infrastructure for massive live traffic.
• Co-design with Applied ML Research for next-gen generative recommendation systems.
• Design automated fault-detection and resilient systems for thousands of GPUs.
• Collaborate with data platforms to enable secure, scalable data processing.

Skills

Python

C++/Java

Distributed systems

GPU clusters

Kubernetes

Slurm

CUDA

PyTorch/TensorFlow

Profiling

Debugging

Education

Bachelor’s or Master’s in CS/CE

Tools

Megatron

DeepSpeed

TensorRT

Triton

KV Cache

Prefill-Decoding

Job description

TikTok USDS Joint Venture LLC is seeking experienced ML engineers to lead and evolve the large-scale real-time training and inference platforms powering its TikTok recommendations. You will drive architecture for multi-node GPU clusters, ensure low-latency serving under heavy traffic, and collaborate with ML researchers to push generative recommendation capabilities in production.

The role requires deep experience with PyTorch/TensorFlow, CUDA, and distributed systems, plus leadership to mentor