Senior AI Infrastructure Engineer: LLM & Recs at Scale

TikTok; San Jose (CA); On-site; USD 128,000 - 317,000; Full time

Benefits offered by this job

Medical, Dental, Vision

401(k) with match

Parental leave

Disability insurance

Life insurance

Wellbeing benefits

Paid holidays

Paid sick days

Paid personal time

Job summary

TikTok is seeking experienced Software Engineers / ML Systems Engineers to join our Model Infrastructure team in San Jose. You will help build the next generation of AI infrastructure powering our For You recommender system and foundation models, handling training, serving, and optimization at massive scale.

As part of a cross-disciplinary team, you will optimize GPU usage, design distributed training pipelines, and collaborate with researchers to bring state-of-the-art models into production

Qualifications

• Bachelor's, Master's, or Ph.D. in CS/CE or related field, or equivalent practical experience.
• 3+ years of software engineering, ML systems, distributed systems experience.
• Proficient in C++ or Python and building production-quality software.
• Experience with PyTorch, TensorFlow, or similar ML frameworks.
• Experience designing large-scale production systems.
• Strong problem-solving and independent project ownership.

Responsibilities

• Design, develop, and optimize large-scale AI training and online inference infrastructure for recommendation models and LLMs.
• Drive architecture of distributed training and serving systems with high scalability, reliability, and efficiency.
• Optimize end-to-end model performance across GPU compute, memory, networking, and runtime systems.
• Develop and optimize LLM training and serving infrastructure, including model parallelism, KV Cache, Continuous Batching, and efficient inference.
• Work closely with researchers and algorithm engineers to productionize new model architectures and algorithms.
• Identify and resolve performance bottlenecks across the full AI stack, from model and framework to GPU kernels and distributed runtime.
• Improve system latency, throughput, GPU utilization, scalability, and infrastructure cost efficiency.
• Drive technical design, implementation, performance benchmarking, and production rollout of critical infrastructure components.
• Mentor junior engineers and contribute to the team's technical direction and engineering standards.

Skills

C++

Python

Distributed systems

Problem solving

GPU optimization

Education

Bachelor's/Master/PhD in CS or related field

Tools

PyTorch

TensorFlow

CUDA

Triton

KV Cache

Job description

TikTok is seeking experienced Software Engineers / ML Systems Engineers to join our Model Infrastructure team in San Jose. You will help build the next generation of AI infrastructure powering our For You recommender system and foundation models, handling training, serving, and optimization at massive scale.

As part of a cross-disciplinary team, you will optimize GPU usage, design distributed training pipelines, and collaborate with researchers to bring state-of-the-art models into production