LLM Inference Systems Engineer

ByteDance; San Jose (CA); On-site; USD 128,000 - 256,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401(k) match

Parental leave

Paid holidays

Wellbeing benefits

Job summary

ByteDance's Volcano Ark MaaS platform invites talented engineers to advance large-model inference systems across China and internal products. You will optimize inference performance, reduce costs, and contribute to stable, scalable compute for Douyin, Toutiao, and Xigua ecosystems.

Join a team that blends cutting-edge ML with practical systems, working on disaggregated inference, KV Cache, and multi-node deployments. Bachelor's/Master's in CS and strong C++/Python skills required.

Qualifications

• Bachelor's or Master's in CS or related field.
• Strong algorithms, design patterns, and data structures.
• Proficient in C++ or Python; good coding style.
• Familiar with CUDA and GPU performance analysis.
• Interested in distributed systems and large-scale inference.

Responsibilities

• Develop Volcano Ark MaaS inference system with focus on performance and stability.
• Reduce inference costs using disaggregated inference, KV Cache, and multi-tenant strategies.

Skills

C++

Python

Algorithms

GPU

Distributed systems

Education

Bachelor's degree in CS

Master's degree in CS

Tools

CUDA

Kubernetes

Ray

RDMA

Job description

ByteDance's Volcano Ark MaaS platform invites talented engineers to advance large-model inference systems across China and internal products. You will optimize inference performance, reduce costs, and contribute to stable, scalable compute for Douyin, Toutiao, and Xigua ecosystems.

Join a team that blends cutting-edge ML with practical systems, working on disaggregated inference, KV Cache, and multi-node deployments. Bachelor's/Master's in CS and strong C++/Python skills required.