Senior ML Infrastructure Engineer — Real-Time, Large-Scale AI

TikTok USDS Joint Venture; San Jose (CA); On-site; USD 187,000 - 360,000; Full time

Job summary

TikTok USDS Joint Venture LLC is seeking an experienced ML infrastructure engineer to own the design and implementation of large-scale training and inference platforms powering TikTok's recommendation and search engines.

You will collaborate with Applied ML researchers and data platform teams to build resilient, low-latency systems across multi-GPU clusters, focusing on 3D parallelism, model serving, and production stability.

Qualifications

• Bachelor's or Master's degree in Computer Science, Computer Engineering, or related technical field.
• 4+ years of software engineering in Python/C++/Java.
• 2+ years building ML infrastructure at enterprise scale.
• Deep familiarity with PyTorch/TensorFlow and GPU memory management (CUDA).
• Experience with production-grade LLM training/inference tools and profiling.

Responsibilities

• Own the roadmap and implementation of large-scale ML training/inference platforms.
• Design multi-node distributed training systems with 3D parallelism (Data, Tensor, Pipeline).
• Build low-latency, high-throughput model serving and inference pipelines.
• Collaborate with Applied ML Research to co-design generative systems for production.
• Implement fault-tolerance and secure, compliant compute stacks.

Skills

Python

C++/Java

ML infra

Distributed training

Profiling

Education

Bachelor's or Master's in CS/Engineering

Tools

PyTorch

TensorFlow

CUDA

InfiniBand

DeepSpeed

Megatron

Job description

TikTok USDS Joint Venture LLC is seeking an experienced ML infrastructure engineer to own the design and implementation of large-scale training and inference platforms powering TikTok's recommendation and search engines.

You will collaborate with Applied ML researchers and data platform teams to build resilient, low-latency systems across multi-GPU clusters, focusing on 3D parallelism, model serving, and production stability.