Senior Software Engineer - Cluster Networking

NVIDIA Corporation

United States

Remote

USD 184,000 - 357,000

Full time

44 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Equity

Job summary

NVIDIA is seeking a senior networking engineer to lead the network architecture of its GPU superclusters, spanning multi-cloud deployments and thousands of nodes.

You will own Kubernetes networking, design overlay topologies, and collaborate with cloud providers to scale AI workloads at scale. The role requires deep Linux networking knowledge and experience with Calico, Tailscale, and WireGuard.

NVIDIA offers equity and benefits and opportunities to impact frontier AI research.

Qualifications

  • BS/MS in Computer Science, Electrical Engineering or related field, or equivalent experience.
  • 6+ years of professional experience in systems, network or infrastructure software engineering.
  • Deep command of Kubernetes networking architecture and CNI standards, with production experience operating Calico strongly preferred.
  • Proficiency designing and maintaining modern mesh and VPN networking topologies - Tailscale, WireGuard or equivalent.
  • Strong Linux networking fundamentals: routing, netfilter and iptables/nftables, packet marking, network namespaces, and how these interact with container runtimes.
  • Demonstrated ability to debug distributed network problems at scale - packet capture, tracing, and correlating behaviour across many hosts to find a single root cause.
  • Proficiency in Go, Python, C or a comparable systems language.
  • Clear written and verbal communication, and the ability to work effectively with engineers across multiple time zones.

Responsibilities

  • You will be the technical owner of how our clusters communicate internally and externally. This includes the CNI data plane, the overlay mesh, and the nodes connecting clusters across regions and providers.
  • Own and evolve the Kubernetes networking architecture for GPU clusters running at multi-thousand-node scale
  • Design, operate and scale the overlay network - CNI, mesh and VPN topologies (Tailscale, WireGuard), and the gateways that connect control and data planes
  • Design, operate and scale the L7 gateways/load balancers/tunnels (Envoy, Cloudflare)
  • Find and eliminate scale ceilings: packet loss under load, control-plane saturation, IP address management exhaustion, and the failure modes that only appear above a few thousand nodes
  • Build the scale-test environments and validation suites that let us catch networking regressions before they reach production, rather than during a training run
  • Diagnose hard, ambiguous problems across the stack - where a symptom in Slurm or a training job traces back to a mark collision, a stale route, or a saturated tunnel
  • Partner with cloud and neocloud providers on network topology, requirements and capabilities as we bring up new clusters
  • Provide senior technical judgement to a distributed team, and depth in the Custer Networking domain

Skills

Kubernetes networking
CNI data plane
Overlay mesh design
Linux networking fundamentals
Go
Python
Networking troubleshooting

Education

BS/MS in Computer Science or Electrical Engineering

Tools

Calico
Tailscale
WireGuard
Envoy
Cloudflare
Kubernetes

Job description

NVIDIA is a pioneer in accelerated computing, known for inventing the GPU and driving breakthroughs in gaming, computer graphics, high-performance computing, and artificial intelligence. Our technology powers everything from generative AI to autonomous systems, and we continue to shape the future of computing through innovation and collaboration.

Within this mission, our team, Managed AI Research Superclusters (MARS), builds and scales the infrastructure, platforms, and tools that enable researchers and engineers to develop the next generation of AI/ML systems.

By joining us, you'll help build solutions that power some of the most sophisticated computing workloads globally. We are looking for a senior networking engineer to lead the network architecture of our GPU superclusters. Our platform runs frontier model training across tens of thousands of GPUs on multiple clouds, and we are scaling it toward clusters of ten thousand nodes and beyond. At that size, networking stops being a configuration exercise and becomes the hardest engineering problem on the platform — and it is currently one of the areas where we most need depth.

What you’ll be doing:
  • You will be the technical owner of how our clusters communicate internally and externally. This includes the CNI data plane, the overlay mesh, and the nodes connecting clusters across regions and providers.
  • Own and evolve the Kubernetes networking architecture for GPU clusters running at multi-thousand-node scale
  • Design, operate and scale the overlay network - CNI, mesh and VPN topologies (Tailscale, WireGuard), and the gateways that connect control and data planes
  • Design, operate and scale the L7 gateways/load balancers/tunnels (Envoy, Cloudflare)
  • Find and eliminate scale ceilings: packet loss under load, control-plane saturation, IP address management exhaustion, and the failure modes that only appear above a few thousand nodes
  • Build the scale-test environments and validation suites that let us catch networking regressions before they reach production, rather than during a training run
  • Diagnose hard, ambiguous problems across the stack - where a symptom in Slurm or a training job traces back to a mark collision, a stale route, or a saturated tunnel
  • Partner with cloud and neocloud providers on network topology, requirements and capabilities as we bring up new clusters
  • Provide senior technical judgement to a distributed team, and depth in the Custer Networking domain
What we need to see:
  • BS/MS in Computer Science, Electrical Engineering or a related field, or equivalent experience
  • 6+ years of professional experience in systems, network or infrastructure software engineering
  • Deep command of Kubernetes networking architecture and CNI standards, with production experience operating Calico strongly preferred
  • Proficiency designing and maintaining modern mesh and VPN networking topologies - Tailscale, WireGuard or equivalent
  • Strong Linux networking fundamentals: routing, netfilter and iptables/nftables, packet marking, network namespaces, and how these interact with container runtimes
  • Demonstrated ability to debug distributed network problems at scale - packet capture, tracing, and correlating behaviour across many hosts to find a single root cause
  • Proficiency in Go, Python, C or a comparable systems language
  • Clear written and verbal communication, and the ability to work effectively with engineers across multiple time zones
Ways to stand out from the crowd:
  • Direct experience architecting and operating massive-scale Kubernetes topologies across thousands of concurrent nodes
  • Experience with high-performance fabrics in AI or HPC environments - InfiniBand, RoCE, or RDMA over converged networks
  • Upstream contributions to Calico, Cilium, Tailscale, or Kubernetes networking SIGs
  • Experience operating networking across multiple public clouds and on-premises environments simultaneously
  • Familiarity with Slurm or other HPC schedulers running on Kubernetes

Your base salary will be determined based on your location, experience, and other objective job-related factors. The expected base salary ranges for this position are: 184,000 USD – 287,500 USD for Level 4; and 224,000 USD – 356,500 USD for Level 5. You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until November 23, 2026. This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Software Engineer - Cluster Networking
Senior Software Engineer - Cluster Networking

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 184,000 - 357,000
Equity
Benefits
Senior HPC AI Cluster Engineer
Senior HPC AI Cluster Engineer

NVIDIA Corporation • United States

Remote
USD 176,000 - 334,000
Senior Network Solution Architect - AI Fabrics
Senior Network Solution Architect - AI Fabrics

NVIDIA • United States

Remote
USD 184,000 - 357,000
Equity
Remote-friendly
Travel opportunities
Senior Software Engineer, Subnet Management
Senior Software Engineer, Subnet Management

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 184,000 - 357,000
Senior Networking Software Architect
Senior Networking Software Architect

Nvidia Corporation in • Santa Clara (CA)

On-site
USD 184,000 - 357,000
Equity
Benefits
Senior Network Engineer - DGX Cloud
Senior Network Engineer - DGX Cloud

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 168,000 - 334,000
Equity
Benefits
Senior Network Solution Architect – AI Fabrics
Senior Network Solution Architect – AI Fabrics

NVIDIA Corporation • Santa Clara (CA)

Remote
USD 184,000 - 357,000
Equity
Benefits
Remote-friendly
+1
Senior Scale-Up Network System Architect
Senior Scale-Up Network System Architect

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 184,000 - 357,000
Equity
Benefits
Senior Software Engineer, AI Networking
Senior Software Engineer, AI Networking

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 184,000 - 357,000
Equity compensation
Benefits
Senior Software Engineer, AI Networking
Senior Software Engineer, AI Networking

NVIDIA Corporation • Austin (TX)

On-site
USD 224,000 - 357,000
Equity
Benefits