Incident Commander — HPC Data Center Operations
(100% Remote | Direct-Hire)
Optomi, in partnership with an AI cloud provider, is seeking a highly capable Incident Commander to lead critical operations support in a high-performance computing (HPC) data center environment. This role leads coordinated responses to high-severity incidents, major outages, and service degradation across HPC infrastructure and customer-facing production systems.
The Incident Commander is the operational authority during major incidents, driving rapid detection, coordinated technical response, executive communication, service restoration, and post-incident improvement. The ideal candidate combines strong incident leadership with a Site Reliability Engineering (SRE) background and experience supporting AI, GPU, or large-scale compute infrastructure.
Key Responsibilities:
- Serve as Incident Commander and operational lead during P1/P2 incidents, major outages, and critical service degradation events.
- Lead structured incident bridges and coordinate cross-functional technical teams through investigation, mitigation, and recovery.
- Monitor major incident queues, escalations, and operational priorities.
- Drive rapid Mean Time to Know (MTTK), service impact assessments, and restoration activities.
- Coordinate recovery validation and confirm service stability before incident closure.
- Own executive communications during high-severity incidents, providing clear, concise, business-focused updates.
- Partner with Problem Management, Change Enablement, SRE, Infrastructure Operations, and Production Engineering teams.
- Facilitate blameless post-incident reviews focused on learning and continuous improvement.
- Use observability, monitoring, and telemetry platforms to improve incident detection, triage, and response.
- Identify opportunities to automate incident detection, escalation, communication, and workflow orchestration.
- Develop and maintain operational playbooks, communication templates, escalation procedures, and response workflows.
- Support operational governance, service dependency mapping, and incident management maturity.
- Apply SRE practices—including service-level objectives, reliability metrics, and error-budget concepts—to incident response and service restoration.
Key Qualifications:
- Bachelor’s degree in Computer Science, Information Technology, Data Science, Statistics, or equivalent hands‑on experience.
- 5+ years of experience in Incident Management, Major Incident Management, SRE, Reliability Operations, Production Operations, NOC/IOC Operations, or similar environments.
- Proven experience leading high-severity incident response in enterprise-scale, highly available production environments.
- SRE experience supporting large-scale infrastructure, cloud platforms, HPC, or production services.
- Experience with AI/GPU infrastructure, accelerator-based compute, GPU clusters, AI training or inference platforms, or high-performance data centers.
- Strong understanding of ITIL Incident Management principles and operational governance.
- Experience coordinating cross-functional technical teams during critical events without direct management authority.
- Familiarity with Service Configuration Management (CMDB) concepts and service dependency mapping.
- Experience supporting operational automation, orchestration, and continuous improvement.
- Hands‑on experience with Jira Service Management or a comparable ITSM platform.
- Strong analytical, organizational, and decision‑making skills in fast‑paced operational environments.
- Excellent verbal and written communication skills, including the ability to engage technical teams and executive stakeholders.
Candidates should bring experience from organizations that build, operate, or depend on large-scale AI and high-performance computing infrastructure.
- AI accelerator, GPU, and semiconductor hardware companies
- GPU cloud providers and AI infrastructure platforms supporting model training and inference
- Hyperscalers and cloud providers operating large compute clusters
- AI model developers and research organizations with substantial compute needs
- Data center, HPC, and AI server operators
- Networking, power, and cooling providers supporting high-density data centers
- Direct experience with GPU cloud providers or AI data center operators is especially relevant, particularly where the role involved production reliability, incident response, or service restoration across large compute clusters.