Graduate Cloud-Native AI Inference Engineer

ByteDance; Seattle (WA); On-site; USD 154,000 - 301,000; Full time

Job summary

ByteDance’s Inference Infrastructure team is hiring engineers to design and operate cloud-native, GPU-accelerated ML platforms at scale. You will work on open-source oriented systems for large-scale LLM inference, contributing to scheduling, resource management, and GPU orchestration in a hyper-scale environment.

We seek PhD graduates with strong distributed systems knowledge, experience with Docker/Kubernetes, and proficiency in Go, Rust, Python, or C++.

Qualifications

• PhD in Software Development, Computer Science, Computer Engineering or related field.
• Strong understanding of large model inference and distributed/parallel systems.
• Hands-on experience building cloud or ML infrastructure (resource management, scheduling, monitoring).
• Solid knowledge of containers and orchestration technologies (Docker, Kubernetes).
• Proficiency in Go, Rust, Python, or C++.

Responsibilities

• Design and build large-scale container-based cluster management and orchestration systems with extreme performance, scalability, and resilience.
• Architect next-generation cloud-native GPU and AI accelerator infrastructure to deliver cost-efficient and secure ML platforms.
• Collaborate across teams to deliver world-class inference solutions using vLLM, SGLang, TensorRT-LLM, and other LLM engines.
• Stay current with the latest advances in open source (Kubernetes, Ray, etc.), AI/ML and LLM infrastructure, and systems research; integrate best practices into production systems.
• Write high-quality, production-ready code that is maintainable, testable, and scalable.

Skills

PhD in software development/computer-

Large model inference

Cloud/ML infrastructure

Docker & Kubernetes

Go/Rust/Python/C++

Education

PhD in Computer Science/Engineering or related field

Tools

Docker

Kubernetes

Ray

TensorRT-LLM

Job description

ByteDance’s Inference Infrastructure team is hiring engineers to design and operate cloud-native, GPU-accelerated ML platforms at scale. You will work on open-source oriented systems for large-scale LLM inference, contributing to scheduling, resource management, and GPU orchestration in a hyper-scale environment.

We seek PhD graduates with strong distributed systems knowledge, experience with Docker/Kubernetes, and proficiency in Go, Rust, Python, or C++.