Senior Server Operations & Maintenance Engineer

ByteDance; San Jose (CA); On-site; USD 122,000 - 272,000; Full time

Job summary

ByteDance is seeking a server operation and maintenance expert to lead NPI and online quality operations for large-scale servers. You will define operation baselines, plan hardware monitoring, and collaborate with internal/external teams to diagnose complex failures and optimize maintainability.

You will also build a knowledge base, advance automation (AIOps), and write technical docs, while traveling internationally up to four times per year as needed.

Qualifications

• Experience managing large-scale servers in the Internet industry.
• Familiar with server hardware product lifecycle and maintenance.
• Proficient in Linux system administration and scripting.

Responsibilities

• Lead operation management for server new product introduction and maintenance baselines.
• Develop hardware monitoring plans and ensure accurate fault detection.
• Troubleshoot complex online server hardware issues with cross-functional teams.
• Build knowledge base and write operation manuals; contribute to monitoring automation.

Skills

Server operations

Linux

BMC/IPMI

Shell scripting

Python

Hardware monitoring

Risk assessment

Education

Bachelor’s degree in Computer Engineering/Computer Science

Tools

Redfish

IPMI/SMBIOS

Job description

Responsibilities

Global server management-Server NPI&Quality Management Team, Responsible for the operation, maintenance management and technical services of the server throughout its life cycle. The job responsibilities cover directions such as the introduction of new product operation and maintenance, operation adaptation like operation and maintenance supervision and control, online quality operation management, the standardization of operation processes, and the planning and construction of platform systems.

We are looking for a server operation and maintenance expert to be responsible for the management of new product introduction and online quality operation., your responsibilities will include:

1. Responsible for the operation management of server new product introduction and the construction of an adaptation system. This includes formulating server operation and maintenance standard baselines, server hardware monitoring and formulation, planning, designing, iteratively optimizing the maintainability, ease of maintenance, and stability solutions for new products, identifying and evaluating operation risks during new product introduction, and constructing and adapting new product operation and maintenance solutions.
2. Deeply involved in formulating hardware monitoring plans and verifying and launching monitoring rules for new online models, new hardware, and new scenarios. Precisely define and continuously improve various hardware failure judgment criteria, enhance the accuracy and recall rate of hardware monitoring, and ensure the stable operation of servers.
3. For complex hardware troubleshooting of online servers, conduct in - depth analysis based on server - level black - box logs (such as BMC logs, IPMI/SMBIOS data, operating system kernel logs, hardware event logs, etc.). Collaborate with internal and external software and hardware R & D experts to accurately locate the root causes of difficult problems. Continuously improve the clarity rate of hardware failure monitoring and diagnosis and the consistency rate between repair reports and actual repairs, effectively reducing the failure return rate.
4. Build a hardware failure knowledge base, accumulate typical cases and troubleshooting methodologies, promote the upgrade of monitoring automation and intelligence (such as the implementation of AIOps anomaly detection), participate in the construction of the hardware monitoring system and the standardization of hardware operation and maintenance processes, and write and maintain technical documents and operation manuals.
5. Online quality operation, Establish unified quality standards, clarify the basis for batch quality determination, define the collaborative interface for batch quality issue repair, accumulate historical batch issue experience, implement risk management in advance, and control risks from the source.
6. International travel requirement: up to four times per year, including but not limited to China, Europe, and South Asia. Candidates must have a valid passport and be able to obtain the necessary visas.

Qualifications

Minimum Qualifications

1. Bachelor’s degree or higher in Computer Engineering, Computer Science or related majors.
2. Have rich experience in the operation and maintenance of large - scale servers in the Internet industry, be familiar with the Internet operation and maintenance process system, and be familiar with the operation management of the server throughout its life cycle. Possess the concept of service expansion and innovation and experience in operation service expansion.
3. Familiar with the development and delivery processes of server hardware products, as well as product development lifecycle management, including product line planning, product phase management, and product document management. Be acquainted with server operation and maintenance scenarios, proficiently master the ability to identify risks related to server maintainability, ease of maintenance, and stability, and possess the experience and ability to adapt to the operation of new server product introductions.
4. Familiar with server hardware architecture, covering x86/ARM architecture, PCIe/NVMe protocols, as well as key components such as CPU, Disk, Memory, GPU/NPU, NIC, and AOC. Familiar with BMC/IPMI protocol, Redfish, Sensor, SEL log, BMC OneKeyLog, GPU stress test diagnosis, and capable of proficiently applying relevant technologies for hardware status monitoring and fault detection.
5. Proficient in the Linux system, skilled in setting server hardware configurations and troubleshooting under the Linux system. Proficient in one of the commonly used computer languages such as Shell scripting, Python, PHP, Perl, or Lua, and able to enhance work efficiency through language tools.

Preferred Qualifications

1. Familiar with the full life cycle management process of servers. Candidates with 3+ years of experience in hardware research and development and testing are preferred.
2. Familiar with server quality management work. 3+ years of experience in online quality management is preferred.
3. Familiar with server hardware monitoring work. Candidates with 3+ years of experience in server hardware index collection, reporting, and monitoring strategy threshold operation are preferred.

About Us

Founded in 2012, ByteDance’s mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok, Lemon8, CapCut and Pico as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

Why Join ByteDance

Inspiring creativity is at the core of ByteDance’s mission. Our innovative products are built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and enrich life - a mission we work towards every day.
As ByteDancers, we strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our Company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Reasonable Accommodation

ByteDance is committed to providing reasonable accommodations in our recruitment processes for candidates with disabilities, pregnancy, sincerely held religious beliefs or other reasons protected by applicable laws. If you need assistance or a reasonable accommodation, please reach out to us at https://tinyurl.com/RA-request

Job Information

【For Pay Transparency】Compensation Description (Annually)

The base salary range for this position in the selected city is $121600 - $272000 annually.

Compensation may vary outside of this range depending on a number of factors, including a candidate’s qualifications, skills, competencies and experience, and location. Base pay is one part of the Total Package that is provided to compensate and recognize employees for their work, and this role may be eligible for additional discretionary bonuses/incentives, and restricted stock units.

Benefits may vary depending on the nature of employment and the country work location. Employees have day one access to medical, dental, and vision insurance, a 401(k) savings plan with company match, paid parental leave, short-term and long-term disability coverage, life insurance, wellbeing benefits, among others. Employees also receive 10 paid holidays per year, 10 paid sick days per year and 17 days of Paid Personal Time (prorated upon hire with increasing accruals by tenure).

The Company reserves the right to modify or change these benefits programs at any time, with or without notice.

For Los Angeles County (unincorporated) Candidates:

1. Interacting and occasionally having unsupervised contact with internal/external clients and/or colleagues;
2. Appropriately handling and managing confidential information including proprietary and trade secret information and access to information technology systems; and
3. Exercising sound judgment.