LLM Backend Engineer: High-Throughput Cloud-Native Systems

ByteDance; San Jose (CA); On-site; USD 120,000 - 170,000; Full time

Job summary

ByteDance is seeking talented engineers to join Volcano Ark MaaS, a large model service platform. You will contribute to end-to-end services including model inference, evaluation, fine-tuning, AI application development, and a robust plugin ecosystem.

Join a team in San Jose focused on backend and cloud-native engineering, building high-availability systems, and advancing large-scale model inference pipelines.

Qualifications

• Bachelor's or Master's in Computer Science or related field.
• Proficient in at least one programming language (Golang, Java, C++, Python) with strong CS fundamentals.
• Understanding of backend engineering systems, databases, networks, OS, and distributed systems.

Responsibilities

• Design and develop core components of Volcano Ark MaaS platform, supporting APIs for text dialogue, multimodal understanding and generation.
• Contribute to cloud-native architecture evolution; implement high-availability features like grayscale releases, traffic shaping, circuit breaking and rate limiting.
• Tune performance for long connections, high throughput, streaming, and low latency in multimodal model inference.
• Ensure stability for large-scale model invocation and resolve bottlenecks during traffic spikes.

Skills

Golang

Java

C++

Python

Education

Bachelor's or Master's in CS or related

Tools

Kubernetes

Docker

Istio

Envoy

Job description

ByteDance is seeking talented engineers to join Volcano Ark MaaS, a large model service platform. You will contribute to end-to-end services including model inference, evaluation, fine-tuning, AI application development, and a robust plugin ecosystem.

Join a team in San Jose focused on backend and cloud-native engineering, building high-availability systems, and advancing large-scale model inference pipelines.