Tech Lead — Scalable ML Infra & Inference

TikTok USDS Joint Venture; Seattle (WA); On-site; USD 198,000 - 416,000; Full time

Job summary

TikTok USDS Joint Venture LLC is seeking a seasoned software engineer for leading large-scale ML training and inference systems powering TikTok recommendations. You will drive architecture, optimize distributed training across GPUs, and collaborate with research and data teams to deliver scalable, secure platforms.

The role emphasizes real-time ML platforms, multi-node parallelism, and performance profiling across complex GPU clusters, with a strong focus on reliability and compliance in a

Qualifications

• Bachelor’s or Master's degree in Computer Science, Computer Engineering, or a related technical discipline.
• 5+ years of professional software engineering experience with deep expertise in Python, C++/Java and a proven track record of designing large-scale distributed systems.
• 3+ years of direct experience building and maintaining machine learning infrastructure at enterprise scale (managing large GPU clusters, Kubernetes, or native Slurm environments).
• Deep technical familiarity with the internals of core ML frameworks (PyTorch / Tensorflow) and a strong understanding of low-level GPU memory management, CUDA interactions, and networking topologies (InfiniBand/RoCE).
• Solid understanding of production-grade LLM training and inference tools, with hands-on profiling skills to eliminate I/O, compute, or network bottlenecks.
• Strong system-level troubleshooting and debugging skills, with experience profiling and eliminating I/O, compute, or network bottlenecks.

Responsibilities

• Technical Leadership: Drive the technical roadmap for large-scale distributed real-time ML training and inferencing platforms with business impact.
• Large-Scale Parallelism Architecture: Architect and scale multi-node distributed training systems and 3D parallelism strategies.
• Production Inference & Serving: Build low-latency, high-throughput model serving infrastructure for massive live traffic.
• Algorithm-Infra Co-Design: Partner with Applied ML Research to co-design generative recommendation systems.
• Resiliency & Fault Tolerance: Design automated fault-detection and checkpointing across thousands of GPUs.
• Cross-Functional Collaboration: Work with researchers and data teams on high-throughput data processing and storage engines.
• Security & Compliance Hardening: Enforce encryption, ACLs, and tenant isolation across compute stack.

Skills

Python

C++/Java

Distributed systems

GPU clusters

Kubernetes

Slurm

CUDA

PyTorch/TensorFlow

Education

Bachelor’s or Master’s degree in Computer Science/Engineering

Tools

Megatron

DeepSpeed

CUDA

Triton

Job description

TikTok USDS Joint Venture LLC is seeking a seasoned software engineer for leading large-scale ML training and inference systems powering TikTok recommendations. You will drive architecture, optimize distributed training across GPUs, and collaborate with research and data teams to deliver scalable, secure platforms.

The role emphasizes real-time ML platforms, multi-node parallelism, and performance profiling across complex GPU clusters, with a strong focus on reliability and compliance in a