Senior Network Reliability Engineer — Production & Automation

ProducePay; San Francisco (CA); On-site; USD 165,000 - 200,000; Full time

Benefits offered by this job

Competitive compensation

RSUs

Paid time off

Healthcare

401(k) Retirement plan with company-m

Tuition reimbursement

Job summary

Crusoe Cloud is seeking a Senior Network Production Operations Engineer to bolster production reliability across our global infrastructure, including edge, backbone, data centers, and GPU interconnects.

The role emphasizes incident response, RCA, and automation-driven execution to keep AI workloads running at scale, with on-call duties and cross-team collaboration across SRE and architecture.

Qualifications

• 5+ years of production network engineering experience in large-scale environments.
• Strong Python and scripting proficiency for diagnostic tooling and automation.
• Hands-on observability tooling experience (Grafana/Prometheus/Kentik/ThousandEyes).
• Experience with data-center networking: BGP, EVPN-VXLAN, OSPF/IS-IS, MPLS, QoS, TCP/IP.
• Experience operating RDMA/RoCE fabrics for GPU/HPC workloads.

Responsibilities

• Production Reliability: support uptime across Crusoe's global network (edge, backbone, data centers, GPU clusters).
• Operational Automation: write Python tooling to reduce toil and accelerate remediation.
• Incident Response: participate in high-severity network events and contribute to postmortem docs.
• Root Cause Analysis: perform RCAs and drive remediation items to completion.
• Observability: build automation on top of monitoring stacks to surface signals faster.
• Operational Standards: improve runbooks, escalation playbooks, and SOPs.

Skills

Python

Incident response

Networking

On-call support

RDMA/RoCE fabrics

BGP/OSPF/IS-IS

Education

Bachelor's degree in CS/EE

Tools

Grafana

Prometheus

Kentik

ThousandEyes

SNMP/NetFlow

Job description

Crusoe Cloud is seeking a Senior Network Production Operations Engineer to bolster production reliability across our global infrastructure, including edge, backbone, data centers, and GPU interconnects.

The role emphasizes incident response, RCA, and automation-driven execution to keep AI workloads running at scale, with on-call duties and cross-team collaboration across SRE and architecture.