Large Model Algorithm Researcher (Multimodal & Code AI)- Soaring Star Talent Program

ByteDance; Singapore; On-site; SGD 180,000 - 280,000; Full time

Job summary

ByteDance's AI Innovation Center is advancing AI infrastructure and cutting-edge research across LLMs and multimodal models. The team explores multilingual content understanding and processing massive video data to improve user experiences and content consumption.

The role emphasizes building robust AI systems, publishing research, and collaborating with engineers and product teams to push the boundaries of AI capabilities within a fast-growing tech company.

Qualifications

• Hold a PhD degree in ML/CV/NLP or related fields.
• Excellent programming skills in C/C++ or Python; strong data structures and algorithms.
• Research experience in large-scale language models (LLMs) and generative AI.

Responsibilities

• Research and develop AI infrastructure for LLMs and multimodal models.
• Advance multilingual understanding and video content understanding capabilities.
• Collaborate with cross-functional teams to deliver R&D breakthroughs.

Skills

C/C++

Python

Algorithms

Competitive programming

Education

PhD in ML/CV/NLP or related

Job description

Responsibilities

Team introduction:

The AI Innovation Center is a department focused on building AI infrastructure and driving cutting-edge research in AI. We explore industry-leading AI technologies, including large language models (LLMs) and multimodal large models, with the goal of developing models that can understand multilingual content and vast amounts of video data, ultimately delivering a better content consumption experience for users. In the Code AI domain, we leverage the powerful code understanding and reasoning capabilities of LLMs to enhance program performance and R&D efficiency.

Project Introduction

Multimodal foundation large models (VLM) represent a research hotspot in the industry and a critical technology for business scenario applications. In 2024, Innovation Center developed VFM V1, a multimodal large model tailored for TikTok's business scenarios. It matches the performance of the best open-source model Qwen VL on public test sets, while significantly outperforming all other foundation models on business test sets. In the future, we aim to continuously develop foundation models with efficient perception and reasoning capabilities, capable of handling multilingual and massive video content understanding algorithms to deliver a better content consumption experience for users.

Project Challenges

• Enhance the multimodal perception encoder: The current encoder uses a fixed frame rate. We need to explore more efficient adaptive frame rates while considering the integration of modalities such as audio and user behavior.
• How to fuse multimodal perception and thinking capabilities to promote stronger comprehensive perception and cognitive abilities of the model.

Qualifications

Minimum Qualifications:

• Hold a PhD degree, with priority given to those who have published papers in fields such as machine learning (ML), computer vision (CV), and natural language processing (NLP).
• Possess excellent programming skills, data structure, and algorithm skills, proficient in C/C++ or Python programming languages. Priority will be given to those who have won awards in competitions such as ACM/ICPC, NOI/IOI, Top Coder, and Kaggle.
• Have research experience in the field of machine learning, particularly in large-scale language models (LLMs) and generative artificial intelligence.

Preferred Qualifications

• Be passionate about technology, have outstanding problem analysis and solving abilities, be enthusiastic about solving challenging problems, and possess good communication skills and team spirit.

Job Information

About Us

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok, Lemon8, CapCut and Pico as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

Why Join ByteDance

Inspiring creativity is at the core of ByteDance's mission. Our innovative products are built to help people authentically express themselves, discover and connect - and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and enrich life - a mission we work towards every day.

As ByteDancers, we strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our Company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.