Multimodal AI Researcher — Vision & Language Models

TikTok Inc.; San Jose (CA); On-site; USD 254,000 - 588,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401(k) with company match

Paid parental leave

Short-term disability

Long-term disability

Life insurance

Wellbeing benefits

Paid holidays

Paid sick leave

Paid Personal Time

Job summary

TikTok Inc. Foundations and Intelligence Service R&D is expanding and seeks researchers in LLM, VLM and Omni Model domains with experience in pretraining, evaluation, data processing and RL-based alignment to push state-of-the-art foundations for real-world TikTok applications.

You will lead research, collaborate with cross-functional teams, optimize multimodal models with OCR/captioning features, and explore efficient architectures to scale downstream TikTok services.

Qualifications

• PhD in CS/DS/AI or related field.
• Experience with single-/multi-modality LLM/VLM pretraining and applications.
• Proficiency in Python, Rust or C++ and DL frameworks (PyTorch, DeepSpeed, Megatron, vLLM).

Responsibilities

• Lead, collaborate and execute on research that pushes forward state-of-the-art in multimodal reasoning and generation research
• Enhance VLM with features like OCR and captioning for TikTok business applications
• Explore model architecture and inference-efficient design to enable scaled downstream TikTok applications
• Work closely with cross-functional teams to plan and implement projects harnessing VLMs for diverse purposes and domains
• Extend insights from industry to academia

Skills

Python

Rust

C++

PyTorch

DeepSpeed

Megatron

vLLM

Education

Ph.D in Computer Science, Data Science, AI or related field

Job description

TikTok Inc. Foundations and Intelligence Service R&D is expanding and seeks researchers in LLM, VLM and Omni Model domains with experience in pretraining, evaluation, data processing and RL-based alignment to push state-of-the-art foundations for real-world TikTok applications.

You will lead research, collaborate with cross-functional teams, optimize multimodal models with OCR/captioning features, and explore efficient architectures to scale downstream TikTok services.