Senior Network Operations Engineer

Cotidal; Palo Alto (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Visa sponsorship

Competitive salary and equity

Relocation support

Job summary

Cotidal Limited is seeking a Senior Network Operations Engineer to keep our data center networks and backbone highly available, reliable, and high-performing. You will operate and troubleshoot production networks, lead incident response, and drive improvements in monitoring, automation and runbooks.

You will work with Juniper routing/switching, Arista EOS, Palo Alto firewalls, and high-performance Ethernet fabrics for AI/HPC workloads.

Qualifications

• 10-15 years of professional network engineering and operations experience.
• Strong production experience with Junos and Juniper MX/QFX/EX routing and switching, plus hands-on Arista EOS operations.
• Advanced troubleshooting of BGP, OSPF, IS-IS, MPLS, EVPN/VXLAN, ECMP, TCP/IP, IPv4/IPv6, LAG/LACP and spine-leaf fabrics.
• Working knowledge of GPU cluster networking, AI/HPC networking, RDMA/RoCEv2, PFC, ECN, QoS and congestion management.
• Hands-on Palo Alto and Juniper SRX firewall operations, including policy troubleshooting, NAT, VPNs, high availability and segmentation.
• Experience with SNMP, syslog, streaming telemetry, packet captures, Wireshark/tcpdump and Linux networking.
• Familiarity with Python, Ansible, Git, incident management, root-cause analysis and change control.
• The ability to resolve complex production incidents independently and communicate clearly across teams.

Responsibilities

• Operate data center fabrics and backbone networks. Manage Juniper MX, QFX and EX platforms and Arista EOS switches.
• Troubleshoot BGP, OSPF, IS-IS, MPLS, EVPN/VXLAN and ECMP across AI/HPC networks.
• Keep high-performance fabrics healthy; monitor RDMA/RoCEv2 operations, PFC, ECN, QoS.
• Lead production incident response; diagnose failures and drive RCAs.
• Carry out software upgrades, capacity expansions and change control.
• Develop automation, dashboards, runbooks and on-call support materials.

Skills

Junos/Juniper MX/QFX/EX routing and

Arista EOS operations

Palo Alto & Juniper SRX firewalls

BGP/OSPF/IS-IS/MPLS/EVPN/VXLAN/ECMP

RDMA/RoCEv2; PFC; ECN; QoS

100G/400G/800G fabric knowledge

Telemetry, SNMP, syslog, Wireshark/tcp

Python, Ansible, Git

Incident management & root-cause分析

Change control and on-call support

Tools

Junos

Arista EOS

Palo Alto firewall

Job description

WHO WE ARE

Supercomputing built for AI.

Cotidal is building the computing infrastructure that AI companies and research labs need to train and run their models. Starting with TPUs, we design the hardware and software together around their workloads, with the goal of making powerful AI chips easier to use at scale.

Our mission is endless compute for humanity. Over time, we want customers to choose across chip stacks without rebuilding their infrastructure.

THE ROLE

We're hiring a Senior Network Operations Engineer to keep Cotidal's data center networks and backbone available, reliable and performing well. You will operate and troubleshoot production networks, lead incident response, and turn what you learn into better monitoring, automation and operating procedures.

You will bring hands-on production experience with Juniper and Arista routing and switching, Palo Alto and Juniper firewalls, and high-performance Ethernet fabrics supporting AI/HPC and GPU workloads. You will work across network, compute, storage and platform engineering, with responsibility for diagnosing complex failures and carrying fixes through to production.

WHAT YOU WILL DO

• Operate data center fabrics and backbone networks. Manage Juniper MX, QFX and EX platforms and Arista EOS switches. Troubleshoot BGP, OSPF, IS-IS, MPLS, EVPN/VXLAN and ECMP across AI/HPC network environments.

• Keep high-performance fabrics healthy. Support RDMA/RoCEv2 operations, including PFC, ECN and QoS. Investigate congestion, packet loss, latency and performance degradation in AI/HPC and GPU cluster networks.

• Operate network security infrastructure. Maintain Palo Alto next-generation firewalls and Juniper SRX environments, including security policies, NAT, IPsec VPNs and high availability.

• Lead production incident response. Diagnose difficult failures, manage operational escalations, lead root-cause analysis and follow through on corrective actions.

• Make safe changes and watch their impact. Monitor telemetry, interface errors, optics, utilization and network health across 100G/400G/800G infrastructure. Carry out software upgrades, maintenance, capacity expansions and deployments through change control.

• Build repeatable operations. Develop automation, monitoring dashboards, runbooks and operational documentation, and participate in on-call support.

WHAT WE ARE LOOKING FOR

• 10-15 years of professional network engineering and operations experience.

• Strong production experience with Junos and Juniper MX/QFX/EX routing and switching, plus hands-on Arista EOS operations.

• Advanced troubleshooting of BGP, OSPF, IS-IS, MPLS, EVPN/VXLAN, ECMP, TCP/IP, IPv4/IPv6, LAG/LACP and spine-leaf fabrics.

• Working knowledge of GPU cluster networking, AI/HPC networking, RDMA/RoCEv2, PFC, ECN, QoS and congestion management. You can isolate packet drops, microbursts, high latency and performance degradation.

• Hands-on Palo Alto and Juniper SRX firewall operations, including policy troubleshooting, NAT, VPNs, high availability and segmentation.

• Experience with SNMP, syslog, streaming telemetry, packet captures, Wireshark/tcpdump and Linux networking.

• Familiarity with Python, Ansible, Git, incident management, root-cause analysis and change control.

• The ability to resolve complex production incidents independently and communicate clearly across teams.

ESPECIALLY VALUABLE

Experience in one or more of the following:

• Operating hyperscale, cloud or AI/HPC environments, including NVIDIA Spectrum Ethernet, InfiniBand or AMD GPU infrastructure.

• An Optical Circuit Switching (OCS) layer, including optical circuit provisioning, path management, monitoring, fault isolation and integration with data center fabrics.

• NVIDIA NCCL, distributed GPU workloads and RDMA performance tuning.

• SONiC, Prometheus, Grafana or Splunk.

• Juniper JNCIP/JNCIE, Palo Alto PCNSE or equivalent certifications.

WHY JOIN COTIDAL

Help establish how we run the networks that connect our AI infrastructure. Your work will link production reliability, performance and safe expansion, with the opportunity to build operating practices that future deployments can use.

Help shape the foundations of an early company through direct discussion and work across network, systems, storage and platform engineering.

LOCATION & WHAT WE OFFER

Location: Palo Alto, in person.

On-call: Participate in production network on-call support.

Compensation: Competitive salary and equity.

Visa sponsorship: We sponsor work visas and help you navigate the process.

Benefits: Medical, dental, and vision coverage, unlimited PTO, and relocation support as needed.