Senior AI Cluster Networking & Performance Engineer

Cisco; San Jose (CA); On-site; USD 168,000 - 245,000; Full time

Benefits offered by this job

Medical, dental and vision insurance

401(k) with Cisco matching

Parental leave

Flexible vacation time off

Volunteer time off

Job summary

Cisco is seeking a highly skilled systems developer to join the CVIS team validating AI infrastructure in the United States. You will investigate kernel-level failures, optimize networking and GPU workloads, and drive end-to-end performance across RDMA, NCCL, and PCIe components.

You will read hardware traces, packets, logs, and NVIDIA tools traces, collaborating with hardware and software teams to eliminate bottlenecks and ensure clusters run at peak efficiency.

Qualifications

• 7+ years of related experience with Linux kernel, networking, GPU, and distributed-systems debugging.
• Experience writing code in C/C++ and Python, with driver development experience.
• Proficient with Linux network stack, RDMA/RoCEv2, NCCL, PCIe, and performance profiling.
• Experience analyzing packet captures, traces, and logs to reproduce benchmarks.

Responsibilities

• Investigate kernel-level failures, boot-time anomalies, PCIe link issues, driver behavior, and network latency.
• Diagnose and tune RDMA/RoCEv2, NCCL, SmartNIC/DPU, NIC queues, congestion, throughput, and collective communications.
• Read and correlate unified hardware traces across host CPUs, PCIe switches, SmartNICs, and GPU compute streams.
• Capture and analyze Wireshark packets, system logs, NVIDIA support logs, traces, profiles, and observability data.
• Partner with hardware and software teams to eliminate end-to-end bottlenecks.

Skills

Linux kernel

Networking

GPU

Distributed systems debugging

C/C++

Python

Device drivers

Education

Bachelor's degree

Master's degree

PhD

Tools

Nsight profiling tools

NVIDIA diagnostic tooling

DOCA SDK

Observability platforms

Job description

Cisco is seeking a highly skilled systems developer to join the CVIS team validating AI infrastructure in the United States. You will investigate kernel-level failures, optimize networking and GPU workloads, and drive end-to-end performance across RDMA, NCCL, and PCIe components.

You will read hardware traces, packets, logs, and NVIDIA tools traces, collaborating with hardware and software teams to eliminate bottlenecks and ensure clusters run at peak efficiency.