Graduate ML Systems Engineer – Kubernetes & GPU Orchestration

ByteDance; San Jose (CA); On-site; USD 128,000 - 256,000; Full time

Job summary

ByteDance is hiring for the Data-AML-Engine Orchestration team to build large-scale ML infrastructure powering online model serving across ByteDance products, including TikTok.

You will design orchestration, scheduling, and resource management systems connecting heterogeneous compute with production ML workloads, with opportunities to focus on GPU utilization, latency, reliability, and MLE productivity.

Qualifications

• Bachelor's or Master's in CS/SE/AI or related field.
• Experience in Go, C++, or Python with solid CS fundamentals.
• Familiarity with Linux, OS concepts, networks, and distributed systems.
• Hands-on mindset; explore systems via logs, metrics, profiling, experiments.
• Demonstrated ownership and collaboration through projects or internships.

Responsibilities

• Design and build foundational orchestration capabilities for ML platforms (Kubernetes Operators, containers, lifecycle management).
• Build multi-tenant resource and quota systems to improve GPU utilization and cost efficiency.
• Develop lifecycle orchestration for online model serving, including deployment, autoscaling, and disaster recovery.
• Create serving orchestration and traffic management for disaggregated clusters with QoS/SLA controls.

Skills

Go

C++

Python

Linux

Distributed systems

Problem solving

Ownership

Education

Bachelor's or Master's in CS/Software Engineering/AI

Tools

Kubernetes

Container runtimes

Volcano

OpenKruise

KServe

Ray Serve

Job description

ByteDance is hiring for the Data-AML-Engine Orchestration team to build large-scale ML infrastructure powering online model serving across ByteDance products, including TikTok.

You will design orchestration, scheduling, and resource management systems connecting heterogeneous compute with production ML workloads, with opportunities to focus on GPU utilization, latency, reliability, and MLE productivity.