Lead Engineer, AI Compute Infrastructure

ByteDance; San Jose (CA); On-site; USD 244,800 - 450,000; Full time

Benefits offered by this job

Day‑one health benefits

401(k) with company match

Parental leave

Disability coverage

Job summary

ByteDance is seeking a Tech Lead Software Engineer to shape AI compute infrastructure in San Jose, driving large‑scale inference platforms and cloud‑native orchestration across multi‑cloud datacenters.

You’ll work on Kubernetes‑native control planes for LLM workloads, collaborate with global teams, and contribute to production‑grade systems using cutting‑edge ML tooling and open‑source technologies.

Qualifications

• BS/MS in Computer Science, Computer Engineering, or related fields with 5 years of relevant experience (PhD with strong systems/ML publications also considered).
• Strong understanding of large model inference, distributed and parallel systems, and/or high‑performance networking systems.
• Hands‑on experience building cloud or ML infrastructure in areas such as resource management, scheduling, request routing, monitoring, or orchestration.
• Solid knowledge of container and orchestration technologies (Docker, Kubernetes).
• Proficiency in Go, Rust, Python, or C++.

Responsibilities

• Design and build large‑scale, container‑based cluster management and orchestration systems with extreme performance, scalability, and resilience.
• Architect next‑generation cloud‑native GPU and AI accelerator infrastructure to deliver cost‑efficient and secure ML platforms.
• Collaborate across teams to deliver world‑class inference solutions using vLLM, SGLang, TensorRT‑LLM, and other LLM engines.
• Stay current with the latest advances in open source (Kubernetes, Ray, etc.), AI/ML and LLM infrastructure, and systems research; integrate best practices into production systems.
• Write high‑quality, production‑ready code that is maintainable, testable, and scalable.

Skills

Large model inference

Distributed systems

High-performance networking

Cloud/ML infrastructure

Docker

Kubernetes

Go

Rust

Python

C++

Education

B.S./M.S. in CS/CE or related fields

Tools

Docker

Kubernetes

CUDA

TensorRT-LLM

vLLM

Job description

ByteDance is seeking a Tech Lead Software Engineer to shape AI compute infrastructure in San Jose, driving large‑scale inference platforms and cloud‑native orchestration across multi‑cloud datacenters.

You’ll work on Kubernetes‑native control planes for LLM workloads, collaborate with global teams, and contribute to production‑grade systems using cutting‑edge ML tooling and open‑source technologies.