Senior SRE: AI Training & Inference Infra

ByteDance

San Jose (CA)

On-site

USD 254,000 - 480,000

Full time

13 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Medical, dental, and vision insurance
401(k) with company match
Paid parental leave
Disability coverage
Life insurance
Paid holidays

Job summary

The ByteDance Seed Infrastructures team is seeking a Senior Site Reliability Engineer to own the reliability of large-scale AI training and inference systems in San Jose. You will drive performance optimization, resource scheduling, and network/storage architecture for heterogeneous GPU clusters.

You will collaborate with ML, product, and operations teams to build robust MLOps solutions, implement monitoring and automated remediation, and help forecast capacity and cost targets for growth.

Qualifications

  • Bachelor's degree or above in Computer Science, Software Engineering or a related field, with 3+ years of experience in large-scale ML/AI infrastructure.
  • Proficient in Python and/or Go; strong Linux development and operations; expert in shell scripting.
  • Expert in containerization (Docker / Kata) and orchestration (Kubernetes) with hands-on experience deploying large-scale compute clusters.
  • Familiar with large-scale high-speed networking (RDMA) and high-performance storage; practical optimization experience.
  • Expert in designing resource scheduling systems and improving utilization; understands scheduling explainability.

Responsibilities

  • Own the stability of training and heterogeneous inference systems for large-scale AI workloads; optimize performance and resolve bottlenecks.
  • Design and iterate a flexible, efficient resource scheduling system with cross-workload support.
  • Lead deployment and operation of large-scale high-speed networks (RDMA) and heterogeneous compute clusters; ensure topology and bandwidth.
  • Own architecture and operation of high-performance storage systems; ensure data reliability and I/O performance.
  • Own end-to-end capacity delivery; plan and optimize resource expansion to meet growth.
  • Build a unified compute pool and establish an operating framework for efficiency and cost control.
  • Develop monitoring, alerting, and automated remediation across training and inference paths.
  • Partner with ML/ops teams to deliver MLOps solutions and raise technical bar.

Skills

Python
Go
Linux
Shell scripting
Docker
Kubernetes
RDMA
Performance optimization
Distributed systems
Network tuning

Education

Bachelor's degree in Computer Science, Software Engineering or related field

Tools

Docker
Kubernetes
Kata
RDMA

Job description

The ByteDance Seed Infrastructures team is seeking a Senior Site Reliability Engineer to own the reliability of large-scale AI training and inference systems in San Jose. You will drive performance optimization, resource scheduling, and network/storage architecture for heterogeneous GPU clusters.

You will collaborate with ML, product, and operations teams to build robust MLOps solutions, implement monitoring and automated remediation, and help forecast capacity and cost targets for growth.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Infra SRE: Reliability & Automation
Senior Data Infra SRE: Reliability & Automation

ByteDance • San Jose (CA)

On-site
USD 218,000 - 388,000
Research Engineer, Large-Scale AI Training Infra
Research Engineer, Large-Scale AI Training Infra

ByteDance • San Jose (CA)

On-site
USD 254,000 - 480,000
Senior AI Infra Engineer - SRE for Inference (Equity)
Senior AI Infra Engineer - SRE for Inference (Equity)

Google LLC • Sunnyvale (CA)

On-site
USD 200,000 - 280,000
Bonus
Equity
Benefits
Senior SRE - Remote, GPU & FinOps Infra Lead
Senior SRE - Remote, GPU & FinOps Infra Lead

Level AI • Mountain View (CA)

Remote
USD 180,000 - 260,000
AI Infra SRE: Build Reliable GPU & Cluster Systems
AI Infra SRE: Build Reliable GPU & Cluster Systems

Boson AI • United States

Remote
USD 125,000 - 250,000
Remote Senior GPU Infrastructure SRE for AI Scale
Remote Senior GPU Infrastructure SRE for AI Scale

Andromeda Cluster • United States

Remote
USD 180,000 - 280,000
Senior SRE: AI Infra on-Site in SF, GPU & Cloud
Senior SRE: AI Infra on-Site in SF, GPU & Cloud

The Recruiting Guy • Arlington (VA)

On-site
USD 175,000 - 250,000
Senior SRE: AI Infra, Hybrid Cloud & Automation
Senior SRE: AI Infra, Hybrid Cloud & Automation

d-Matrix • Santa Clara (CA)

On-site
USD 140,000 - 210,000
Senior AI Inference Engineer — Scale & Production
Senior AI Inference Engineer — Scale & Production

Spectro Cloud • San Jose (CA)

On-site
USD 140,000 - 210,000
Senior SRE: AI Infra & GPU Clusters
Senior SRE: AI Infra & GPU Clusters

Spacex • Palo Alto (CA)

Hybrid
USD 165,000 - 265,000
Stock options
401(k) plan
Medical, vision, and dental coverage
+2