Graduate Backend Inference Framework Engineer - Scalable AI

BYTEDANCE PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

ByteDance is seeking a skilled engineer to architect and implement a scalable model inference system for large-parameter AI models. You will address challenges across deployment scenarios, optimize inference framework core modules, and drive improvements in performance, latency, and resource utilization.

The role requires handling high-concurrency distributed services, staying current with inference tech, and collaborating across teams to deliver system-wide improvements.

Qualifications

• Bachelor's or Master's in computing or related field.
• Solid C/C++ programming skills and knowledge of data structures and algorithms.
• Familiar with multi-threaded concurrency and basic performance tuning in multi-threaded scenarios.
• Experience in R&D of high-concurrency distributed services and awareness of latency/resource optimization.
• Good learning, execution, and cross-team collaboration skills.

Responsibilities

• Design and implement architecture for high-performance model inference services.
• Optimize core modules of the inference framework (scheduling, monitoring, canary release).
• Track and adopt new inference technologies and promote standardization of the team’s tech system.

Skills

C/C++ programming

Linux

Concurrency

Performance tuning

Distributed services

Education

Bachelor's or Master's in computing

Tools

Redis

RocksDB

BRPC

GRPC

Job description

ByteDance is seeking a skilled engineer to architect and implement a scalable model inference system for large-parameter AI models. You will address challenges across deployment scenarios, optimize inference framework core modules, and drive improvements in performance, latency, and resource utilization.

The role requires handling high-concurrency distributed services, staying current with inference tech, and collaborating across teams to deliver system-wide improvements.