Tech Lead — Large‑Scale ML Infrastructure

TikTok USDS Joint Venture; San Jose (CA); On-site; USD 187,000 - 438,000; Full time

Job summary

TikTok USDS Joint Venture is seeking a senior ML/software engineer to lead the technical roadmap for large-scale distributed ML training and inference platforms. The role focuses on scalable GPU clusters, multi-node training, and high-throughput serving in a fully in-person setting.

Responsibilities include co-designing generative recommendation systems, building resilient and low-latency serving pipelines, and collaborating with Applied ML Research and data platform teams.

Qualifications

• Bachelor’s or Master’s in CS/Engineering or related field.
• 5+ years software engineering with distributed systems.
• 3+ years ML infra experience at enterprise scale (GPU clusters, Kubernetes, Slurm).
• Deep familiarity with PyTorch/TensorFlow internals; CUDA knowledge.
• Experience with production-grade LLM training and inference tools.

Responsibilities

• Drive the technical roadmap for large-scale distributed real-time ML training and inference platforms.
• Architect multi-node distributed training with 3D parallelism; manage GPU clusters and pipelines.
• Build low-latency, high-throughput model serving infrastructure and optimize inference pipelines.
• Co-design generative recommendation systems with Applied ML Research.
• Design automated fault-detection and resilient systems for large GPU clusters.
• Collaborate with researchers and data platforms for high-throughput data processing.
• Enforce security/compliance hardening across the compute stack.

Skills

Python

C++/Java

Kubernetes

Distributed systems

LLM training

Education

Bachelor's or Master's in CS/Engineering

Tools

PyTorch

TensorFlow

CUDA

Slurm

Job description

TikTok USDS Joint Venture is seeking a senior ML/software engineer to lead the technical roadmap for large-scale distributed ML training and inference platforms. The role focuses on scalable GPU clusters, multi-node training, and high-throughput serving in a fully in-person setting.

Responsibilities include co-designing generative recommendation systems, building resilient and low-latency serving pipelines, and collaborating with Applied ML Research and data platform teams.