Senior ML Infra Engineer - Real-Time, Large-Scale

TikTok USDS Joint Venture; Seattle (WA); On-site; USD 178,000 - 342,000; Full time

Benefits offered by this job

Medical insurance

Dental & vision

401(k) match

Parental leave

Disability benefits

Wellbeing programs

Paid holidays

Job summary

TikTok USDS Joint Venture LLC is seeking experienced software engineers to build and operate large-scale ML training and inference platforms powering TikTok’s recommendation and search engines. You will work on distributed systems, low-latency model serving, and security-focused data processing in highly scalable environments.

The team emphasizes collaboration with Applied ML Research, scalable GPU infrastructure, and robust fault tolerance to support live global services.

Qualifications

• 4+ years of professional software engineering experience with Python, C++/Java
• 2+ years building ML infrastructure at enterprise scale
• Strong knowledge of PyTorch/TensorFlow and GPU memory management
• Experience with low-level networking topologies (InfiniBand/RoCE) and distributed training
• Experience with production-grade LLM training and inference tools

Responsibilities

• Contribute to technical roadmap and implement large-scale distributed real-time ML training and inference platforms
• Design and scale multi-node distributed training systems with advanced 3D parallelism
• Build and scale low-latency, high-throughput model serving infrastructure
• Co-design generative recommendation systems with Applied ML Research
• Develop fault-detection and recovery mechanisms for large GPU clusters
• Collaborate with data platform teams to ensure secure multi-modal data processing
• Implement encryption-at-rest/in-transit, ACLs, and tenant isolation

Skills

Python

C++/Java

ML infra

PyTorch/TensorFlow

CUDA

Distributed systems

LLM training

Education

Bachelor’s or Master’s in CS/CE

Tools

Megatron

DeepSpeed

Triton

TensorRT

Job description

TikTok USDS Joint Venture LLC is seeking experienced software engineers to build and operate large-scale ML training and inference platforms powering TikTok’s recommendation and search engines. You will work on distributed systems, low-latency model serving, and security-focused data processing in highly scalable environments.

The team emphasizes collaboration with Applied ML Research, scalable GPU infrastructure, and robust fault tolerance to support live global services.