ML Infrastructure Engineer — Real-Time Recs & Search

TikTok USDS Joint Venture; Seattle (WA); On-site; USD 130,000 - 246,000; Full time

Job summary

TikTok USDS Joint Venture LLC is seeking an experienced software engineer to contribute to large-scale distributed ML training and real-time inference platforms powering TikTok's recommendations and search engines.

You will design multi-node training systems, optimize GPU utilization, and collaborate with Applied ML Research to push state-of-the-art generative recommendation capabilities. On-site teams operate fully in-person up to 5 days a week.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related technical discipline.
• 2+ years of professional software engineering experience with deep expertise in Python, C++/Java
• 1+ years of direct experience building and maintaining machine learning infrastructure at enterprise scale
• Deep technical familiarity with the internals of core ML frameworks (PyTorch / TensorFlow) and a strong understanding of low-level GPU memory management, CUDA interactions, and networking topologies (InfiniBand/RoCE).
• Solid understanding of production-grade LLM training and inference tools, with hands-on profiling skills to eliminate I/O, compute, or network bottlenecks.
• Strong system-level troubleshooting and debugging skills, with experience profiling and eliminating I/O, compute, or network bottlenecks.

Responsibilities

• Technical Execution & System Ownership: contribute to the technical roadmap and hands-on implementation of large-scale distributed ML training and inferencing platforms.
• Large-Scale Parallelism Architecture: design and scale multi-node distributed training systems with 3D parallelism (Data, Tensor, Pipeline).
• Production Inference & Serving: build low-latency, high-throughput model serving infrastructure for massive live traffic.
• Algorithm-Infra Co-Design: collaborate with Applied ML Research to co-design next-gen Generative Recommendation systems.
• Resiliency & Fault Tolerance: develop automated fault-detection and asynchronous checkpointing for thousands of GPUs.
• Cross-Functional Collaboration: work with data platform teams to engineer high-throughput data processing and storage engines.
• Security & Compliance Hardening: implement encryption, ACLs, and tenant isolation across the compute stack.

Skills

Python

C++/Java

ML Infrastructure

PyTorch/TensorFlow

GPU/CUDA

LLM Training

Performance Debugging

Education

Bachelor’s or Master’s in CS/CE

Tools

Megatron

DeepSpeed

Triton

TensorRT

CUDA

Job description

TikTok USDS Joint Venture LLC is seeking an experienced software engineer to contribute to large-scale distributed ML training and real-time inference platforms powering TikTok's recommendations and search engines.

You will design multi-node training systems, optimize GPU utilization, and collaborate with Applied ML Research to push state-of-the-art generative recommendation capabilities. On-site teams operate fully in-person up to 5 days a week.