Senior AI Inference Engineer - Multimodal Large Models

TikTok; San Jose (CA); On-site; USD 212,800 - 450,000; Full time

Job summary

TikTok is seeking a Senior AI Infra Engineer to advance its large-model inference systems in San Jose. You will design and optimize the infrastructure for ultra-large-scale language and multimodal models, ensuring scalable, low-latency online serving across heterogeneous compute resources.

Join a team building high-performance kernels and AI-driven automation for production inference. Bring expertise in CUDA, Triton, and large-model optimization to deliver robust, scalable AI infrastructure at

Qualifications

• Bachelor's degree or above in Computer Science, Software Engineering, AI, Mathematics, or related fields.
• 4+ years of experience in high-performance computing, distributed scheduling, or large-model inference engine development.
• Familiarity with large-model architectures and strong system design skills for high-concurrency environments.
• Strong understanding of asynchronous scheduling, resource pooling, and load balancing in distributed microservice systems.
• Strong engineering skills in performance optimization and production system development.

Responsibilities

• Build and evolve next-generation inference systems for large-scale online traffic, including global scheduling across heterogeneous compute resources and high-concurrency load balancing.
• Optimize distributed inference for 200B+ models and complex multimodal models to improve throughput and latency in production.
• Develop high-performance kernels for frontier model architectures using CUDA, Triton, and related tools.
• Explore AI-driven infrastructure for inference systems, including AI Agents for kernel optimization and deployment pipelines.

Skills

HPC

Distributed scheduling

Large-model inference

System design

Load balancing

Performance optimization

Education

Bachelor's degree in Computer Science or related field

Tools

CUDA

Triton

Cutlass

vLLM

SGLang

Job description

TikTok is seeking a Senior AI Infra Engineer to advance its large-model inference systems in San Jose. You will design and optimize the infrastructure for ultra-large-scale language and multimodal models, ensuring scalable, low-latency online serving across heterogeneous compute resources.

Join a team building high-performance kernels and AI-driven automation for production inference. Bring expertise in CUDA, Triton, and large-model optimization to deliver robust, scalable AI infrastructure at