Inference Systems Backend Engineer - ARK Large Model Platform (Singapore)

ByteDance; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

ByteDance Singapore is hiring to advance Volcano Engine's large-model training and inference systems, optimizing performance across thousands of GPUs and managing ultra-large-scale inference traffic. The role focuses on C/C++ and Python on Linux, with experience in distributed ML pipelines, and collaboration with algorithm teams to improve both algorithms and systems.

Expected to explore architectures, hardware integration (GPUs/TPUs), and perform heavy profiling and optimization in a

Qualifications

• Proficiency in C/C++ and Python on Linux for large-scale ML systems.
• Experience with at least one major ML framework (TF, PyTorch, MXNet).
• Familiar with large-model training/inference frameworks (vLLM, TensorRT-LLM, Megatron-LM).
• Strong problem-solving, independence, teamwork and communication.

Responsibilities

• Develop and optimize Volcano Engine large-model training and inference systems, scaling across thousands of GPUs and scheduling large-scale inference traffic.
• Solve challenges related to high concurrency, reliability and scalability for hundreds of billions of tokens.
• Research architectures for training/inference, including subgraph matching, compiler optimization, and model quantization.
• Integrate heterogeneous hardware (GPUs/NPUs/TPUs) with training and inference frameworks.
• Improve compute utilization across globally distributed GPU clusters through elastic scheduling and orchestration.
• Collaborate with algorithm teams to jointly optimize algorithms and systems.

Skills

C/C++

Python

Linux

distributed ML systems

TensorFlow

PyTorch

MXNet

Megatron-LM

vLLM

TensorRT-LLM

Education

Master's degree or PhD in computer science or related field

Job description

Responsibilities

• Develop and performance‑optimize the Volcano Engine large‑model training and inference systems, including model computation optimization, tuning thousands‑GPU training clusters, distributed LLM inference systems, and large‑scale inference traffic scheduling.
• Solve technical challenges related to high concurrency, high reliability, and high scalability, supporting Volcano Engine’s daily training and inference traffic at the scale of hundreds of billions of tokens.
• Research and introduce forward‑looking technical architectures for large‑model training and inference, including subgraph matching, compiler optimization, and model quantization.
• Integrate heterogeneous hardware (GPUs, NPUs, TPUs) with training and inference frameworks.
• Improve compute utilization across globally distributed ultra‑large‑scale GPU clusters through elastic scheduling, GPU oversubscription, task orchestration, and related techniques.
• Collaborate closely with algorithm teams to jointly optimize algorithms and systems.

Qualifications

Minimum Qualifications

• Proficient in C/C++ and Python development under Linux environments, with experience in large‑scale machine‑learning systems or search, advertising, and recommendation systems.
• Familiar with at least one machine‑learning framework (TensorFlow, PyTorch, MXNet, or other in‑house frameworks).
• Familiar with at least one large‑model training or inference framework (vLLM, TensorRT‑LLM, SGLang, Megatron‑LM).
• Strong problem‑solving skills, ability to work independently, excellent teamwork spirit, and outstanding capability in breaking down complex problems.
• Strong sense of responsibility, good learning ability, communication skills, and self‑motivation.

Preferred Qualifications

• Experience designing architectures for large‑scale distributed systems.
• Strong understanding of GPU hardware architecture and GPU software stack (such as CUDA, cuDNN), with experience in GPU performance analysis and optimization.
• Research background at the Master’s or Ph.D. level in computer‑systems‑related fields (distributed systems, parallel computing, programming languages and compilers, networking, or storage systems).