Senior Applied ML Production Engineer

ByteDance; San Jose (CA); On-site; USD 162,000 - 388,000; Full time

Job summary

ByteDance is seeking an experienced engineer to own production stability and reliability for AML training, inference, and storage systems in our San Jose location. You will drive SLOs, observability, and on-call processes while improving CI/CD, canary releases, and auto-rollback capabilities.

The role emphasizes cross-functional collaboration, performance analysis, and automation platform development to optimize resource usage and overall R&D efficiency.

Qualifications

• Bachelor’s degree or above in Computer Science, Software Engineering, Artificial Intelligence, or related fields.
• Familiar with Linux and proficient in at least one language: Shell, Python, Go, or C++.
• Understanding of machine learning training/inference architectures, Kubernetes, GPU clusters, or distributed storage systems.
• Proven experience in online troubleshooting, performance analysis, and building automation platforms.
• Strong sense of responsibility, clear logical thinking, and ability to drive resolution of complex issues across cross-functional teams.

Responsibilities

• System Stability & Production Management for AML training, inference, and storage systems.
• Reliability Engineering: SLO/SLA, observability, alerting, on-call processes, fault diagnosis, auto-healing, disaster recovery, post-mortems.
• Engineering Excellence: CI/CD, canary releases, auto-rollback, automated inspections, pre-flight checks, capacity forecasting, elastic auto-scaling.
• Resource & Cost Management: governance across GPU/CPU/storage/network, quota management, cost attribution, performance tuning.

Skills

Strong sense of responsibility

Clear logical thinking

Cross-functional collaboration

Troubleshooting

Education

Bachelor’s degree or above in Computer Science, Software Engineering, AI, or related fields

Tools

Linux

Kubernetes

GPU clusters

Distributed storage systems

Job description

ByteDance is seeking an experienced engineer to own production stability and reliability for AML training, inference, and storage systems in our San Jose location. You will drive SLOs, observability, and on-call processes while improving CI/CD, canary releases, and auto-rollback capabilities.

The role emphasizes cross-functional collaboration, performance analysis, and automation platform development to optimize resource usage and overall R&D efficiency.