Senior Software Engineer, SRE and Production Engineering - DGX Cloud

Jobgether; Canada; Remote; CAD 140,000 - 190,000; Full time

Benefits offered by this job

Fully remote in Canada

Equity eligibility

Full-time position

Benefits and wellness programs

Job summary

Jobgether seeks a Senior Software Engineer specializing in SRE and Production Engineering to design and operate large-scale GPU infrastructure powering demanding AI workloads.

You will work with bare-metal provisioning, firmware, Linux, and Kubernetes across cloud partner and on-prem environments, driving automation and reliability at scale.

This remote role offers exposure to NVIDIA NVL72 systems, DPUs, and complex production challenges across hardware, networking, and platform engineering teams.

Qualifications

• 12+ years of experience in production infrastructure software or operations.
• Strong Go or Python programming skills for automation and services.
• Experience with BMC/Redfish for provisioning and health checks.
• Hands-on with NVIDIA GPU hardware (NVL72) and BlueField DPUs.
• Strong Linux, firmware/driver management, and server lifecycle knowledge.
• Experience with on-call, incident response, and observability.

Responsibilities

• Develop automation for bare-metal provisioning and cluster lifecycle management.
• Build tools interfacing with BMC/Redfish for hardware health and recovery workflows.
• Operate NVIDIA NVL72 systems and BlueField DPUs across environments.
• Diagnose and resolve complex hardware/software failures.
• Convert recurring infra issues into automated detection and repair.
• Define validation procedures for safe capacity introduction.
• Participate in on-call rotations and RCAs.
• Collaborate across hardware, networking, and data-center teams.
• Improve reliability and automation across the fleet.

Skills

Go

Python

Linux

BMC

Redfish

Kubernetes

Networking

DPU

NVL72

Education

Bachelor's or Master's in Computer Science

Tools

BlueField-3

Job description

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Software Engineer, SRE and Production Engineering - DGX Cloud based in Canada.

Join a team building and operating large-scale GPU infrastructure that powers demanding artificial intelligence workloads. As a Senior Software Engineer specializing in Site Reliability Engineering (SRE) and Production Engineering, you will develop the automation and operational systems that transform bare-metal hardware into reliable, production-ready cloud services. You will work hands‑on with advanced GPU servers, data processing units, firmware, Linux, and Kubernetes across cloud partner and on‑premises environments. Your work will improve infrastructure provisioning, hardware validation, system recovery, and cluster lifecycle management. You will also help strengthen production reliability by investigating complex failures and turning recurring incidents into lasting automated solutions. This remote opportunity offers the chance to contribute to next‑generation AI infrastructure at significant scale.

Accountabilities

Responsibilities

• Develop automation for bare‑metal server provisioning, hardware validation, firmware and software upgrades, repairs, and cluster lifecycle management.
• Build tools that interact with Baseboard Management Controller (BMC) and Redfish interfaces to monitor hardware health, manage server states, and support recovery workflows.
• Operate and advance NVIDIA NVL72 systems and BlueField‑3 or newer Data Processing Units (DPUs) across cloud partner and on‑premises environments.
• Diagnose and resolve complex failures involving servers, DPUs, GPU and CPU systems, networking, Linux, and Kubernetes.
• Convert recurring infrastructure issues into automated detection, troubleshooting, and repair processes.
• Define validation procedures and production handoff criteria to ensure new infrastructure capacity is introduced safely and consistently.
• Participate in on‑call rotations, incident response, root‑cause investigations, and follow‑up activities to implement permanent corrective solutions.
• Collaborate with hardware, networking, platform engineering, data center operations, and partner teams to resolve issues across organizational boundaries.
• Improve infrastructure reliability, operational consistency, and automation across the production fleet.

Requirements

• At least 12 years of experience building software for or operating production infrastructure, including substantial hands‑on bare‑metal experience.
• Strong programming skills in Go or Python, with a proven track record of delivering production automation and services.
• Direct experience using BMC and Redfish interfaces for server provisioning, hardware health inspection, power management, or fault diagnosis.
• Practical experience working directly with NVIDIA GPU hardware, particularly NVL72 systems, and BlueField‑3 or newer DPUs.
• Strong knowledge of Linux, firmware and driver management, network boot processes, and server lifecycle operations from initial provisioning through repair.
• Experience maintaining production reliability through on‑call responsibilities, incident response, observability, and implementation of durable solutions.
• Ability to troubleshoot complex issues across hardware, host operating systems, networking, and distributed services.
• Strong communication skills and demonstrated ownership of technical problems involving multiple teams.
• Bachelor's or Master's degree in Computer Science, or equivalent practical experience.

Additional Experience That Would Be An Advantage

• Experience operating BlueField DPUs in DPU mode, including host‑to‑DPU connectivity and lifecycle troubleshooting.
• Familiarity with NVLink, InfiniBand, Spectrum‑X, or GPU cluster performance validation.
• Experience designing safe and repeatable workflows for rack‑scale deployment, firmware upgrades, hardware replacement, and customer handoffs.
• Knowledge of Kubernetes, GitOps, Argo CD, Service Level Objectives (SLOs), and fleet‑wide automation.

Benefits

• Fully remote work arrangement in Canada.
• Full‑time position.
• Competitive compensation package with eligibility for equity.
• Access to employee benefits and wellness programs.
• Opportunity to work hands‑on with advanced GPU infrastructure and large‑scale AI workloads.
• Exposure to complex production engineering challenges across cloud and on‑premises environments.
• Collaboration with specialized teams working across hardware, networking, platform engineering, and data center operations.
• Opportunity to contribute to the development and operation of next‑generation AI infrastructure.