AI Inference Systems Engineer Intern

TikTok; San Jose (CA); On-site; USD 70,000 - 95,000; Full time

Benefits offered by this job

Health insurance

Life insurance

Wellbeing benefits

Paid holidays

Housing allowance

Job summary

TikTok is offering a PhD internship focused on building and optimizing the inference infrastructure for ultra-large-scale AI systems. You will work on distributed serving, global scheduling, and low-latency inference across heterogeneous compute resources.

Ideal candidates are pursuing a PhD and have strong system design and performance optimization skills, with experience in CUDA, Triton, and distributed microservices.

Qualifications

• Currently pursuing a PhD in Software Development, Computer Science, Computer Engineering, or a related technical discipline.
• Familiarity with large-model architectures and strong system design skills for complex, high-concurrency environments.
• Strong engineering skills in performance optimization and production system development.
• Strong understanding of asynchronous scheduling, resource pooling, and load balancing in distributed microservice systems (preferred).

Responsibilities

• Build and evolve next-generation inference systems for large-scale online traffic, including global scheduling across heterogeneous compute resources, high-concurrency load balancing, and efficient batch formation.
• Optimize distributed inference for 200B+ models and complex multimodal models through TP, EP, DP, and related strategies to improve throughput and latency in production.
• Develop high-performance kernels for frontier model architectures such as MoE, emerging attention mechanisms, and multimodal fusion layers using CUDA, Triton, and related tools.
• Explore AI-driven infrastructure for inference systems, including AI Agents for kernel optimization, performance tuning, consistency validation, deployment pipelines, and intelligent operations.

Skills

Large-model architectures

System design

Performance optimization

Distributed systems

Education

PhD in Software Development/Computer Science/Computer Engineering

Tools

CUDA

Triton

Job description

TikTok is offering a PhD internship focused on building and optimizing the inference infrastructure for ultra-large-scale AI systems. You will work on distributed serving, global scheduling, and low-latency inference across heterogeneous compute resources.

Ideal candidates are pursuing a PhD and have strong system design and performance optimization skills, with experience in CUDA, Triton, and distributed microservices.