Inference Network Engineer (Anycast / GPU Fabric / Fiber)

The Edge Inference; City of Albany (NY); On-site; USD 180,000 - 230,000; Full time

Benefits offered by this job

Equity

Home office stipend

Certification reimbursement

Job summary

Edge Inference Inc. in Albany, NY is seeking a senior Network Engineer to design and operate a nationwide anycast routing fabric for AI inference.

You will own BGP routing, optical transport, and latency measurement across multiple sites, collaborating with architecture teams to deploy automated provisioning for rapid rollouts. The ideal candidate has 7+ years of production BGP experience, hands-on anycast, GPU fabric knowledge, and scripting (Python/Ansible).

Qualifications

• 7+ years of network engineering with deep BGP ownership.
• Hands-on anycast experience across many PoPs.
• Knowledge of GPU east-west fabrics (InfiniBand/RoCEv2/Spectrum-X).
• Optical/transport experience: 400G/800G, DWDM, IRU or lit-service.
• IX peering experience and PeeringDB familiarity.
• Fluency with network automation and treating configs as code.

Responsibilities

• Design and operate BGP anycast routing to land requests at nearest healthy Inference Point.
• Engineer GPU east-west fabrics and lossless Ethernet configurations.
• Standardize optical transport and dark-fiber strategies for site parity.
• Build/own dark fiber ring architecture and peering programs.
• Measure latency (p50/p95/p99) by site/prefix and publish results.
• Support customer deployments with network architectures and PNIs.
• Provide tier-3 escalation for latency SLA issues and fiber cuts.
• Automate deployment so new points turn up from templates.

Skills

BGP ownership

Anycast design

GPU fabric knowledge

Latency engineering

IP routing policy

Network automation

Bachelor's degree in CS/Networking

Education

Bachelor's degree in Computer Science, Network Engineering, or related field

Tools

Python

Ansible

Job description

Edge Inference Inc. is building the nationwide AI inference network — a distributed fabric of Inference Points that puts GPU compute within single-digit milliseconds of where inference is actually consumed. The entire thesis rests on the network: an Inference Point is only worth building if a user's request lands at the nearest healthy site, traverses a GPU fabric fast enough to keep tokens flowing, and returns inside the latency SLA. Our commitment is — that is a routing and fiber problem before it is anything else.

About the Role

The Inference Network Engineer owns three layers at once: BGP anycast inference routing that steers every request to the nearest healthy Inference Point, the GPU east-west fabric inside each site, and the dark fiber and peering that connect each IP to its parent Hub and to the internet. You define the network standard inside the IP Standard Design, deploy it identically across every Deployment Wave, and prove the latency numbers with measurement rather than assertion. This is a builder's role on a network that does not exist yet. You will design the anycast architecture, negotiate the fiber, turn up the peering, and instrument the whole thing so the NIOC can see TTFT and inter-token latency by site, by prefix, and by customer.

Responsibilities

• Design and operate the BGP anycast inference routing architecture so every request lands at the nearest healthy Inference Point — including AS and prefix strategy, health-driven route withdrawal on GPU or thermal degradation, and controlled failover to the parent Spoke, Satellite, or Hub.
• Engineer the GPU east-west fabric standard inside each IP: InfiniBand NDR/XDR, NVIDIA Spectrum-X Ethernet, and RoCEv2 designs with lossless configuration, congestion control, adaptive routing, and rail-optimized topologies for GB200/GB300 NVL72 and HGX H200 deployments.
• Standardize the optical layer — 400G/800G optics, transceiver and breakout matrices, structured cabling, and pathway requirements — as a frozen section of the IP Standard Design so every site is cabled identically.
• Build and own the dark fiber ring architecture connecting each Inference Point to its parent Hub, including route diversity, conduit entry separation, IRU negotiation, and lit-service fallback where dark fiber is not yet economic.
• Run the IX peering and private network interconnect program: establish peering at major exchanges reachable from Hub sites, negotiate PNIs with the eyeball networks and AI platforms that drive inference traffic, and manage PeeringDB presence and peering policy.
• Own latency engineering and measurement: build the active and passive measurement fabric that reports p50/p95/p99 latency, TTFT contribution, and inter-token latency by site and by prefix, and publish the results against the
• Define the north-south edge design at each IP — border routers, DDoS mitigation, load balancing to inference gateways, and traffic engineering between anycast catchments during Deployment Wave energizations.
• Implement BGP security and routing hygiene across the network: RPKI/ROA for all Edge Inference prefixes, MANRS compliance, prefix filtering, and route policy that survives a national footprint with 200+ origination points.
• Automate network configuration and turn-up so a new Inference Point can be brought onto the anycast fabric from a template rather than a hand-built config, with configuration state validated continuously.
• Support Inference Solutions Architects and the ICA sales motion with network architecture for customer deployments — multi-IP placement for latency, dedicated PNI, private routing, and sovereignty-constrained traffic paths.
• Provide tier-three network escalation to the NIOC, including root cause analysis for latency SLA excursions, fabric congestion events, and fiber cuts affecting IP reachability.
• Evaluate and qualify network vendors and platforms against the roadmap, and feed network-driven changes back into the next IPSD release.

Qualifications

• 7+ years of network engineering experience with deep, production BGP ownership — eBGP/iBGP design, route policy, communities, traffic engineering, and multi-homing at scale.
• Hands-on anycast experience: designing or operating anycast service delivery across many points of presence, including health-based route control and catchment management.
• Working knowledge of GPU east-west fabrics — InfiniBand, RoCEv2, or Spectrum-X — including lossless Ethernet configuration, congestion control, and the failure modes that stall collective operations.
• Optical and transport experience: 400G/800G optics, DWDM, dark fiber route evaluation, and IRU or lit-service contracting.
• IX peering experience, including PeeringDB, route servers, bilateral settlement-free peering, and PNI negotiation.
• Fluency with network automation — Python, Ansible, or equivalent — and with treating device configuration as versioned artifacts rather than hand edits.
• Bachelor's degree in Computer Science, Network Engineering, or a related field, or equivalent operational experience.

Required Skills

• 7+ years of network engineering experience with deep, production BGP ownership — eBGP/iBGP design, route policy, communities, traffic engineering, and multi-homing at scale.
• Hands-on anycast experience: designing or operating anycast service delivery across many points of presence, including health-based route control and catchment management.
• Working knowledge of GPU east-west fabrics — InfiniBand, RoCEv2, or Spectrum-X — including lossless Ethernet configuration, congestion control, and the failure modes that stall collective operations.
• Optical and transport experience: 400G/800G optics, DWDM, dark fiber route evaluation, and IRU or lit-service contracting.
• IX peering experience, including PeeringDB, route servers, bilateral settlement-free peering, and PNI negotiation.
• Fluency with network automation — Python, Ansible, or equivalent — and with treating device configuration as versioned artifacts rather than hand edits.

Preferred Skills

• CCNP/CCIE (Service Provider or Routing & Switching), JNCIP, or NVIDIA NCP-AIO / NCA-AIIO certification.
• Experience networking rack-scale GPU systems(GB200/GB300 NVL72) or large InfiniBand NDR clusters in production.
• RPKI/ROA deployment and MANRS participation at a network operator.
• Experience with inference-serving traffic patterns — streaming token responses, long-lived connections, KV-cache-aware routing, or gateway-level request steering.
• Kubernetes networking(CNI, service mesh, gateway APIs) in a GPU-serving context.
• Background supporting sovereignty or government network requirements, including constrained routing and data-residency enforcement.

Base salary commensurate with anycast and GPU fabric depth and certification status. Annual performance bonus tied to latency SLA attainment and Inference Points brought onto the anycast fabric. Equity participation in the national rollout. Full benefits including home office equipment allowance for remote employees and certification reimbursement (CCIE/CCNP, JNCIP, NVIDIA NCP-AIO)

Job can be located in the Albany, New York metropolitan area or will consider a remote opportunity for the right candidate.