Senior Site Reliability Engineer (Seed Model - Infrastructures - SRE)

ByteDance; San Jose (CA); On-site; USD 254,000 - 480,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

401(k) with company match

Parental leave

Job summary

ByteDance Seed is hiring a Senior Site Reliability Engineer to own the stability of training and inference platforms powering foundation models. You will optimize resource scheduling, network and storage for large-scale heterogeneous GPU/NPU/XPU clusters, and collaborate with ML/algorithms teams on MLOps best practices.

The role focuses on end-to-end reliability, capacity planning, and cost-conscious compute delivery in a global Seed infra environment.

Qualifications

• Bachelor's degree or above in Computer Science, Software Engineering or related field with 3+ years in ML infra/MLOps or similar.
• Proficient in Python and/or Go; Linux dev/ops, strong shell scripting for automation.
• Expert in containers/orchestration (Docker/Kubernetes) and deploying large scale compute clusters.

Responsibilities

• Own stability of training and heterogeneous inference systems for large scale generative AI workloads.
• Design and iterate a flexible resource scheduling system to improve utilization.
• Lead deployment and optimization of large-scale high-speed networks (RDMA) and heterogeneous compute clusters.
• Own storage architecture for high performance I/O to meet training/inference data needs.
• Develop capacity planning and cost-control frameworks for compute resources.
• Build a unified compute pool for training and inference workloads.

Skills

Python

Go

Linux

Shell scripting

Docker

Kubernetes

RDMA networking

Storage systems

Resource scheduling

Education

Bachelor's degree or higher in CS or related field

Tools

Docker

Kata

Kubernetes

RDMA

Job description

Senior Site Reliability Engineer (Seed Model - Infrastructures - SRE)

Location:

San Jose

Team:

Technology

Employment Type:

Regular

Job Code:

A40565A

Share this listing:

Responsibilities

About the TeamThe Seed Infrastructures team oversees the distributed training, reinforcement learning framework, high-performance inference, and heterogeneous hardware compilation technologies for AI foundation models.The Seed Infra SRE team is responsible for the delivery and stability of the compute infrastructure behind Seed's foundation models. We own the reliability of large-scale heterogeneous GPU clusters, the training and inference platforms that run on them, and the capacity, network and storage systems that keep them running. Our work directly determines how fast and how reliably we can train and serve the models.

• Own the stability of training systems and heterogeneous inference systems for large-scale generative AI workloads (LLM, text-to-image, text-to-video, multimodal understanding); drive deep performance optimization and resolve core issues such as training interruptions, inference latency and throughput bottlenecks.
• Design and iterate a flexible, efficient resource scheduling system; improve scheduling efficiency, resource utilization and scheduling explainability; support fine-grained scheduling and priority management across workload types (training, offline inference, online serving).
• Lead the planning, deployment, operation and optimization of large-scale high-speed networks (RDMA) and heterogeneous compute clusters (GPU / NPU / XPU); ensure sound cluster topology, stable network bandwidth and efficient delivery of compute resources.
• Own the architecture, selection and operation of high-performance storage systems; ensure storage I/O performance, data reliability and stability to meet the data access needs of large-scale training and inference.
• Own end-to-end quality of capacity delivery; build capacity planning methodology, and combine business growth forecasts to complete resource expansion and configuration optimization so that capacity is delivered on time, reliably and cost-effectively.
• Drive the build-out of a unified compute pool with co-located training and inference workloads; improve compute utilization for offline inference and similar scenarios; establish an operating framework for compute efficiency and cost control, and codify reusable methodology.
• Safeguard the stability of complex online services and engineering architecture; build monitoring, alerting and automated remediation that cover the full training and inference path; respond to and resolve complex production issues quickly.
• Partner with algorithm, product and operations teams to deliver MLOps technical solutions; establish engineering best practices for large-scale generative AI, and raise the technical bar of the team.

Minimum Qualifications

• Bachelor's degree or above in Computer Science, Software Engineering or a related field, with 3+ years of experience in large-scale machine learning / generative AI MLOps or closely related infrastructure work.
• Proficient in Python and/or Go; solid Linux development and operations skills; expert in shell scripting and able to independently build automation tooling.
• Expert in containerization (Docker / Kata) and orchestration (Kubernetes), with hands-on experience deploying, operating and optimizing large-scale heterogeneous compute clusters (GPU / NPU).
• Familiar with the principles and operation of large-scale high-speed networking (RDMA) and high-performance storage (distributed / object storage), with practical experience optimizing these systems.
• Expert in the design and optimization of resource scheduling systems, with hands-on experience iterating scheduling policies and improving utilization; understands the key design considerations for scheduling explainability.
• Experience ensuring stability and optimizing performance of large-scale generative AI (LLM / image / video generation) training or inference systems; able to quickly localize and resolve complex issues across the full training and inference path.
• Project experience with co-located training / inference compute pools or offline inference efficiency; understands the core metrics and levers of compute operations. Strong communication, cross-team collaboration and self-drive; a strong sense of ownership; able to operate under the pressure of large-scale production environments.

Preferred Qualifications

• End-to-end experience productionizing large-scale generative AI models (LLM, text-to-image, text-to-video).
• Experience managing heterogeneous compute clusters and capacity planning in multi-cloud or hybrid-cloud environments.
• Familiarity with distributed training frameworks and inference optimization tooling, with hands-on performance tuning experience.
• Experience with large-scale network performance tuning (RDMA bandwidth / latency) or breaking through storage I/O bottlenecks.
• Experience delivering large and complex compute capacity and building the associated operating framework, with quantified results in cost control or utilization improvement.
• Experience designing large-scale distributed system architectures and running fault-injection / chaos exercises.

Job Information

The base salary range for this position in the selected city is $254400 - $480000 annually.

Compensation may vary outside of this range depending on a number of factors, including a candidate’s qualifications, skills, competencies and experience, and location. Base pay is one part of the Total Package that is provided to compensate and recognize employees for their work, and this role may be eligible for additional discretionary bonuses/incentives, and restricted stock units.

Benefits may vary depending on the nature of employment and the country work location. Employees have day one access to medical, dental, and vision insurance, a 401(k) savings plan with company match, paid parental leave, short-term and long-term disability coverage, life insurance, wellbeing benefits, among others. Employees also receive 10 paid holidays per year, 10 paid sick days per year and 17 days of Paid Personal Time (prorated upon hire with increasing accruals by tenure).

The Company reserves the right to modify or change these benefits programs at any time, with or without notice.

For Los Angeles County (unincorporated) Candidates:

Qualified applicants with arrest or conviction records will be considered for employment in accordance with all federal, state, and local laws including the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. Our company believes that criminal history may have a direct, adverse and negative relationship on the following job duties, potentially resulting in the withdrawal of the conditional offer of employment:

1. Interacting and occasionally having unsupervised contact with internal/external clients and/or colleagues;
2. Appropriately handling and managing confidential information including proprietary and trade secret information and access to information technology systems;
3. Exercising sound judgment.

About Doubao (Seed)

Established in 2023, the ByteDance Seed team is dedicated to pioneering new paths toward artificial general intelligence. We aspire to advance the frontier of intelligence to drive progress for both technology and society.

With a long-term vision for the AI sector, the Seed team's research spans MLLM, GenMedia, AI for Science, and Robotics. We maintain a global presence with laboratories and career opportunities across China, Singapore, and the United States. To date, we have launched industry-leading general foundation models and cutting-edge multimodal capabilities. Our technology powers over 50 application scenarios — including Doubao, Jimeng, TRAE, Dola and Dreamnia — and serves enterprise customers through Volcano Engine and BytePlus. Third-party data shows that the Doubao App ranks first in user volume in the Chinese market, while Doubao foundation models lead the industry in average daily token consumption.

Why Join ByteDance

Inspiring creativity is at the core of ByteDance's mission. Our innovative products are built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and enrich life - a mission we work towards every day.

As ByteDancers, we strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our Company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Reasonable Accommodation

ByteDance is committed to providing reasonable accommodations in our recruitment processes for candidates with disabilities, pregnancy, sincerely held religious beliefs or other reasons protected by applicable laws. If you need assistance or a reasonable accommodation, please reach out to us at https://tinyurl.com/RA-request