Backend Inference Engineer — High-Performance GPU

BYTEDANCE PTE. LTD.; Singapore; On-site; SGD 100,000 - 160,000; Full time

Job summary

ByteDance's Data AML team builds training and inference systems for recommendation and advertising. This role is for graduates who want to tackle challenging GPU-accelerated workloads and join a world-class ML platform supporting Douyin, Jinri Toutiao, and Xigua Video.

You will work on optimizing the large-model inference engine, exploring tensor and pipeline parallelism, and benchmarking against leading frameworks.

Qualifications

• Bachelor's or Master's degree in computing or related field.
• Solid foundation in low-level computing, proficient in C/C++ and Python, CUDA programming, GPU architecture knowledge.
• Proficient in deep learning operators and GPU optimization, including memory access and scheduling.
• Familiar with deep learning inference compilation: graph optimization, fusion, quantization.
• Experience with GPU performance tools (Nsight, Profiler) and software-hardware co-optimization.
• Strong collaboration, communication, and project execution skills.

Responsibilities

• Iterate the architecture of the large model inference engine to optimize GPU performance and reduce latency.
• Adapt inference engine to various GPU/NPU hardware architectures and ensure high universality.
• Lead design and optimization of distributed parallel solutions (tensor/pipeline/sequence/MoE) for large models.
• Stay current with large-model inference tech (GPU HPC, distributed parallelism, cache optimization) and benchmark against frameworks like vLLM and TensorRT-LLM.

Skills

C/C++

Python

CUDA programming

GPU architecture

Deep learning inference

Nsight/Profiler

Education

Bachelor's/Master's in computing

Tools

Nsight

Profiler

TensorRT

Job description

ByteDance's Data AML team builds training and inference systems for recommendation and advertising. This role is for graduates who want to tackle challenging GPU-accelerated workloads and join a world-class ML platform supporting Douyin, Jinri Toutiao, and Xigua Video.

You will work on optimizing the large-model inference engine, exploring tensor and pipeline parallelism, and benchmarking against leading frameworks.