Graduate Software Engineer - Cloud-Native GPU Inference

ByteDance; San Jose (CA); On-site; USD 162,000 - 317,000; Full time

Benefits offered by this job

Medical Insurance

Dental Insurance

Vision Insurance

401k with match

Paid parental leave

Disability Insurance

Life Insurance

Wellbeing Benefits

Paid Holidays

Paid Sick Days

Paid Personal Time

Job summary

ByteDance's Inference Infrastructure team is seeking engineers to design, build, and operate cloud-native GPU-accelerated ML infrastructure at scale, including vLLM, SGLang, and TensorRT-LLM work.

You will join a world-class team within Core Compute Infrastructure, contributing to open-source ecosystems, scheduling, and orchestration across multi-cloud environments. A PhD and deep expertise in distributed systems help you drive high-performance inference at scale.

Qualifications

• PhD in Software Development, Computer Science, Computer Engineering, or related discipline.
• Experience with large model inference and distributed systems.
• Hands-on experience building cloud or ML infrastructure incl. resource management, scheduling, monitoring.
• Solid knowledge of container/orchestration tech (Docker, Kubernetes).
• Proficiency in Go, Rust, Python, or C++.

Responsibilities

• Design and build large-scale container-based cluster management and orchestration systems.
• Architect next-generation cloud-native GPU and AI accelerator infrastructure.
• Collaborate to deliver inference solutions using vLLM, SGLang, TensorRT-LLM.
• Stay current with open source and ML infra advances; implement best practices.
• Write high-quality, production-ready code.

Skills

Go

Rust

Python

C++

Docker

Kubernetes

Distributed systems

Cloud infrastructure

Education

PhD in Software Development, Computer Science, Computer Engineering

Tools

Docker

Kubernetes

Job description

ByteDance's Inference Infrastructure team is seeking engineers to design, build, and operate cloud-native GPU-accelerated ML infrastructure at scale, including vLLM, SGLang, and TensorRT-LLM work.

You will join a world-class team within Core Compute Infrastructure, contributing to open-source ecosystems, scheduling, and orchestration across multi-cloud environments. A PhD and deep expertise in distributed systems help you drive high-performance inference at scale.