Senior Site Reliability Engineer (Seed Model - Infrastructures - SRE)

ByteDance; San Jose (CA); On-site; USD 180,000 - 280,000; Full time

Job summary

ByteDance’s Seed Infra SRE team manages the compute infrastructure for foundation models, including distributed training, heterogenous hardware, and high-speed networks. You will own training/inference stability, optimize resource scheduling, and lead deployment of large-scale GPU/NPU clusters while ensuring data reliability and cost-efficient capacity planning.

You will partner with product, algorith m, and operations teams to implement robust MLOps practices, drive performance optimizations,

Qualifications

• Bachelor's degree in CS or related field; 3+ years in large-scale ML/MLOps infra or similar work.
• Proficient in Python and/or Go; strong Linux and shell scripting; automation tooling experience.
• Expert in containerization and orchestration (Docker/Kubernetes); hands-on with large clusters.
• Familiar with high-speed networking (RDMA) and high-performance storage.
• Strong scheduling experience and ability to optimize compute utilization.

Responsibilities

• Own stability of training and hetero inference systems for large-scale AI workloads; optimize performance and resolve bottlenecks.
• Design and iterate resource scheduling systems; improve utilization and explainability; support multi-type workloads.
• Lead deployment and operation of large-scale networks and heterogeneous compute clusters (GPU/NPU); ensure topology and bandwidth.
• Own architecture and operations of high-performance storage; ensure I/O performance and data reliability.
• Drive capacity planning,Compute pool build-out, and cost control for scalable delivery.
• Build monitoring, alerting, and automated remediation across training and inference paths; rapid issue resolution.
• Collaborate with algorithm/product/ops teams to deliver MLOps solutions and raise technical bar.

Skills

Python

Go

Linux

Shell scripting

Distributed systems

Education

Bachelor's degree in CS/related field

Tools

Docker

Kubernetes

Kata containers

RDMA

Job description

About the Team

The Seed Infrastructures team oversees the distributed training, reinforcement learning framework, high-performance inference, and heterogeneous hardware compilation technologies for AI foundation models.
The Seed Infra SRE team is responsible for the delivery and stability of the compute infrastructure behind Seed's foundation models. We own the reliability of large-scale heterogeneous GPU clusters, the training and inference platforms that run on them, and the capacity, network and storage systems that keep them running. Our work directly determines how fast and how reliably we can train and serve the models.

Responsibilities

• Own the stability of training systems and heterogeneous inference systems for large-scale generative AI workloads (LLM, text-to-image, text-to-video, multimodal understanding); drive deep performance optimization and resolve core issues such as training interruptions, inference latency and throughput bottlenecks.
• Design and iterate a flexible, efficient resource scheduling system; improve scheduling efficiency, resource utilization and scheduling explainability; support fine-grained scheduling and priority management across workload types (training, offline inference, online serving).
• Lead the planning, deployment, operation and optimization of large-scale high-speed networks (RDMA) and heterogeneous compute clusters (GPU / NPU / XPU); ensure sound cluster topology, stable network bandwidth and efficient delivery of compute resources.
• Own the architecture, selection and operation of high-performance storage systems; ensure storage I/O performance, data reliability and stability to meet the data access needs of large-scale training and inference.
• Own end-to-end quality of capacity delivery; build capacity planning methodology, and combine business growth forecasts to complete resource expansion and configuration optimization so that capacity is delivered on time, reliably and cost-effectively.
• Drive the build-out of a unified compute pool with co-located training and inference workloads; improve compute utilization for offline inference and similar scenarios; establish an operating framework for compute efficiency and cost control, and codify reusable methodology.
• Safeguard the stability of complex online services and engineering architecture; build monitoring, alerting and automated remediation that cover the full training and inference path; respond to and resolve complex production issues quickly.
• Partner with algorithm, product and operations teams to deliver MLOps technical solutions; establish engineering best practices for large-scale generative AI, and raise the technical bar of the team.

Minimum Qualifications

• Bachelor's degree or above in Computer Science, Software Engineering or a related field, with 3+ years of experience in large-scale machine learning / generative AI MLOps or closely related infrastructure work.
• Proficient in Python and/or Go; solid Linux development and operations skills; expert in shell scripting and able to independently build automation tooling.
• Expert in containerization (Docker / Kata) and orchestration (Kubernetes), with hands-on experience deploying, operating and optimizing large-scale heterogeneous compute clusters (GPU / NPU).
• Familiar with the principles and operation of large-scale high-speed networking (RDMA) and high-performance storage (distributed / object storage), with practical experience optimizing these systems.
• Expert in the design and optimization of resource scheduling systems, with hands-on experience iterating scheduling policies and improving utilization; understands the key design considerations for scheduling explainability.
• Experience ensuring stability and optimizing performance of large-scale generative AI (LLM / image / video generation) training or inference systems; able to quickly localize and resolve complex issues across the full training and inference path.
• Project experience with co-located training / inference compute pools or offline inference efficiency; understands the core metrics and levers of compute operations. Strong communication, cross-team collaboration and self-drive; a strong sense of ownership; able to operate under the pressure of large-scale production environments.

Preferred Qualifications

• End-to-end experience productionizing large-scale generative AI models (LLM, text-to-image, text-to-video).
• Experience managing heterogeneous compute clusters and capacity planning in multi-cloud or hybrid-cloud environments.
• Familiarity with distributed training frameworks and inference optimization tooling, with hands-on performance tuning experience.
• Experience with large-scale network performance tuning (RDMA bandwidth / latency) or breaking through storage I/O bottlenecks.
• Experience delivering large and complex compute capacity and building the associated operating framework, with quantified results in cost control or utilization improvement.
• Experience designing large-scale distributed system architectures and running fault-injection / chaos exercises.