ML Orchestration Engineer for Scalable Model Serving

Bytedance; San Jose (CA); On-site; USD 162,000 - 317,000; Full time

Benefits offered by this job

Medical insurance

401(k) match

Paid parental leave

Wellbeing benefits

Paid time off

Job summary

ByteDance in San Jose, CA, is seeking a highly skilled data systems engineer to join the Data-AML-Engine Orchestration team. You will design and build large-scale ML orchestration, scheduling, and resource management systems that power online model serving across ByteDance products, including TikTok.

You will work on systems affecting GPU utilization, latency, reliability and scalability, with opportunities to focus on Kubernetes operators, multi-tenant scheduling, and disaster recovery as you

Qualifications

• PhD in Computer Science, Software Engineering, AI or related field.
• Proficiency in Go, C, or Python with strong data structures knowledge.
• Familiarity with Linux, OS concepts, networking, concurrency and distributed systems.
• Hands-on, exploratory mindset with ability to read code, metrics, logs and run experiments.
• Systematic, quantitative problem solver with ability to measure and validate improvements.
• Proven ownership and collaboration through coursework, research, internships or open-source projects.

Responsibilities

• Design and build foundational orchestration capabilities for ML platforms (Kubernetes Operators, container runtimes, lifecycle management).
• Build multi-tenant resource and quota systems to improve GPU utilization and cost efficiency.
• Develop lifecycle orchestration for online model serving (distribution, deployment, upgrades, autoscaling, DR).
• Create serving orchestration and traffic management for disaggregated clusters (topology-aware scheduling, QoS/SLA).

Skills

PhD in CS

Go

C

Python

Linux

Distributed Systems

Data structures

Education

PhD

Tools

Kubernetes

Container runtimes

Git

Job description

ByteDance in San Jose, CA, is seeking a highly skilled data systems engineer to join the Data-AML-Engine Orchestration team. You will design and build large-scale ML orchestration, scheduling, and resource management systems that power online model serving across ByteDance products, including TikTok.

You will work on systems affecting GPU utilization, latency, reliability and scalability, with opportunities to focus on Kubernetes operators, multi-tenant scheduling, and disaster recovery as you