ML Infrastructure Lead - Real-Time, Large-Scale Training

TikTok USDS Joint Venture; San Jose (CA); On-site; USD 187,000 - 438,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

401(k) with company match

Paid parental leave

Disability coverage

Life insurance

Paid time off and holidays

Job summary

TikTok USDS Joint Venture LLC is seeking an experienced machine learning infrastructure leader to drive the roadmap for ultra large-scale training and real-time inference across billions-parameter models. You will supervise multi-node GPU clusters, optimize data pipelines, and partner with research teams to push state-of-the-art generative recommendation systems.

The role requires on-site presence in Los Angeles, collaboration across cross-functional teams, and a strong focus on security,

Qualifications

• Bachelor's or Master's in CS/CE or related field.
• 5+ years of professional software engineering with distributed systems.
• 3+ years building ML infrastructure at enterprise scale.
• Deep familiarity with PyTorch/TensorFlow, CUDA, and networking.
• Experience with LLM training/inference tools.

Responsibilities

• Lead technical roadmap for large-scale distributed real-time ML training and inference platforms.
• Architect and scale multi-node distributed training systems with advanced 3D parallelism.
• Build low-latency, high-throughput model serving infrastructure.
• Co-design with Applied ML Research for next-generation generative systems.
• Design automated fault-detection and recovery in multi-thousand GPU clusters.
• Collaborate with data platform teams to ensure secure, scalable data processing.

Skills

Python

C++/Java

Distributed systems

GPU clusters

Kubernetes

PyTorch/TensorFlow

Low-level memory management

Profiling

Education

Bachelor's or Master’s degree in Computer Science or related

Tools

Megatron

DeepSpeed

CUDA

Triton

TensorRT

Job description

TikTok USDS Joint Venture LLC is seeking an experienced machine learning infrastructure leader to drive the roadmap for ultra large-scale training and real-time inference across billions-parameter models. You will supervise multi-node GPU clusters, optimize data pipelines, and partner with research teams to push state-of-the-art generative recommendation systems.

The role requires on-site presence in Los Angeles, collaboration across cross-functional teams, and a strong focus on security,