GPU Infrastructure Engineer

PolarGrid

Ottawa

On-site

CAD 120,000 - 180,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

PolarGrid, an early-stage Canadian company delivering managed AI inference, is seeking a GPU Infrastructure Engineer to scale its distributed GPU fleet across sites. You will own provisioning, burn-in, validation, and staged production rollout for new sites, ensuring smooth traffic shifts and latency control.

You will run the global routing layer, tune performance using live signals like latency and capacity, and build observability, automation, and reliability practices to keep services

Qualifications

  • Experience operating distributed systems across multiple sites or regions with high availability targets.
  • Strong networking fundamentals: BGP, DNS, TCP/TLS and load balancing.
  • Hands-on experience running bare-metal GPU fleets: provisioning, burn-in, firmware and node lifecycle.
  • InfiniBand or RDMA networking experience.
  • Experience running LLM inference engines such as vLLM, SGLang or TensorRT-LLM, including multi-node serving, cold starts and TTFT.
  • Deep observability skills focused on p95/p99 latency and gray failures, not just uptime.
  • Experience designing failover, graceful degradation and automated recovery.
  • Strong automation and infrastructure-as-code mindset, including CI/CD for infrastructure.
  • Ability to plan capacity on constrained, uneven hardware supply.
  • Experience building internal tooling or control planes, ideally in Go or Rust.

Responsibilities

  • Scale PolarGrid's inference network to a geographically distributed fleet of GPU clusters.
  • Own provisioning, burn-in, validation and rollout of new sites.
  • Define site readiness for traffic and ensure low latency.
  • Manage global routing layer for best location routing based on latency, capacity and health.
  • Build observability with end-to-end tracing, latency alerting, and safe staged rollouts.
  • Develop automation and infrastructure-as-code for reliability and recovery.
  • Plan capacity with constrained hardware supply.
  • Operate multi-site, high-availability distributed systems.

Skills

Distributed systems
Networking
Observability
Automation
Infra as code
Go
Rust

Tools

CI/CD for infrastructure
Terraform

Job description

About PolarGrid

PolarGrid is an early-stage Canadian company that runs managed AI inference. A customer brings a model or a serving image. We run it on dedicated GPUs behind one endpoint, serve each request from the nearest site, and move traffic to the next nearest site when one fails.

Our goal is to make AI inference fast, reliable, and secure for customers running production workloads.

The Role

This role is responsible for scaling PolarGrid's inference network from a handful of sites to a geographically distributed fleet of GPU clusters. The GPU Infrastructure Engineer will own how new sites and nodes come online, from provisioning and burn-in through validation and a staged move into production. They will define when a site is ready for customer traffic, and make sure each new site joins the network without hurting latency or reliability.

A core responsibility is running the global routing layer that sends each request to the best place to serve it. The GPU Infrastructure Engineer will work with live signals such as network latency, capacity and health, and tune routing against real traffic to keep latency low and predictable.

The role also owns reliability across the network. The GPU Infrastructure Engineer will set and track availability targets, build the health checks and automation that detect and route around failing hardware and models, keep recovery fast, and test failure handling regularly.

The GPU Infrastructure Engineer will build the observability, release and network systems needed to run a distributed GPU fleet without operations work growing with every new site. This includes end-to-end tracing and latency alerting, safe staged rollouts, network configuration as code, and security and DDoS protection at the edge.

Requirements

  • Experience operating distributed systems across multiple sites or regions with high availability targets

  • Strong networking fundamentals: BGP, DNS, TCP/TLS and load balancing

  • Hands-on experience running bare-metal GPU fleets: provisioning, burn-in, firmware and node lifecycle

  • InfiniBand or RDMA networking experience

  • Experience running LLM inference engines such as vLLM, SGLang or TensorRT-LLM, including multi-node serving, cold starts and TTFT

  • Deep observability skills focused on p95/p99 latency and gray failures, not just uptime

  • Experience designing failover, graceful degradation and automated recovery

  • Strong automation and infrastructure-as-code mindset, including CI/CD for infrastructure

  • Ability to plan capacity on constrained, uneven hardware supply

  • Experience building internal tooling or control planes, ideally in Go or Rust

Nice to Have

  • DDoS mitigation experience

  • Running DNS infrastructure at scale

  • Hardware security: TPM, measured boot or confidential computing

  • Real-time streaming inference, including voice and audio

  • Operating remote datacenter sites with on-site technicians

  • Certificate automation across many domains

  • Deterministic simulation testing

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Global GPU Infra & Inference Network Lead
Global GPU Infra & Inference Network Lead

PolarGrid • Ottawa

On-site
CAD 120,000 - 180,000
Staff Product Engineer
Staff Product Engineer

PolarGrid • Ottawa

On-site
CAD 120,000 - 150,000
Solutions Architect
Solutions Architect

PolarGrid • Toronto

Remote
CAD 90,000 - 150,000
Site Reliability Engineer, AI/ML Infrastructure
Site Reliability Engineer, AI/ML Infrastructure

Boson AI • Toronto

On-site
CAD 100,000 - 130,000
Product Manager
Product Manager

PolarGrid • Ottawa

On-site
CAD 90,000 - 130,000
Senior AI Infra Engineer — GPU, Kubernetes, RDMA (Equity)
Senior AI Infra Engineer — GPU, Kubernetes, RDMA (Equity)

NVIDIA Corporation • Canada

Remote
CAD 130,000 - 190,000
Equity
Benefits
Remote Solutions Architect - GPU-Driven Edge & AI Platforms
Remote Solutions Architect - GPU-Driven Edge & AI Platforms

PolarGrid • Toronto

Remote
CAD 90,000 - 150,000
GPU Cloud Platform Engineer
GPU Cloud Platform Engineer

Yotta Labs • Canada

Remote
CAD 90,000 - 120,000
Flexible work environment
Cutting-edge technology projects
Collaborative team culture
Staff Software Engineer, GPU Inference
Staff Software Engineer, GPU Inference

Cerebras • Toronto

On-site
CAD 150,000 - 210,000
Software Engineer, GPU Inference
Software Engineer, GPU Inference

Cerebras • Toronto

On-site
CAD 140,000 - 210,000