Domain Architect - Ai Storage- 12M Contract - Perm

Northbridge Recruitment; Melbourne; In loco; AUD 180.000 - 240.000; Tempo pieno

Descrizione del lavoro

Northbridge Recruitment is seeking a Domain Architect – AI Storage to design, deploy and optimise HPC and AI storage infrastructure for large-scale GPU clusters and AI factories in Melbourne.

The role leads the storage component of AI deployments, defining repeatable architectures, integrating with Kubernetes/OpenShift, and delivering high-throughput NVMe-oF, GDS and S3 storage solutions, with hands-on delivery and performance tuning.

Competenze

• Extensive experience delivering HPC/AI storage solutions.
• Hands-on with parallel file systems WEKA, Lustre, BeeGFS, GPFS.
• Experience designing storage for GPU clusters and AI workloads.
• Strong Linux storage and performance troubleshooting.
• Automation with Python, Ansible or Terraform.
• Knowledge of NVMe-oF, RDMA and S3/object storage.

Mansioni

• Design, deploy and optimise HPC and AI storage for GPU clusters.
• Lead implementation of parallel file systems such as WEKA, Lustre, BeeGFS and GPFS.
• Architect high-throughput storage fabrics for AI training and inference.
• Implement NVMe-oF, GDS, NFS over RDMA and S3 object storage.
• Integrate storage with bare-metal servers, Linux compute nodes, Kubernetes and OpenShift.
• Configure CSI drivers, storage classes and persistent storage for AI workloads.
• Benchmark and troubleshoot storage performance using IOR, FIO and mdtest.
• Develop automated deployment and configuration processes using Python, Ansible or Terraform.
• Design data lifecycle and tiering strategies across high-performance NVMe, flash and object storage.
• Collaborate with compute and network architects to ensure end-to-end performance.
• Lead technical implementation, performance tuning, testing and validation.

Conoscenze

High-performance storage design

HPC storage architecture

Storage for GPU clusters

Strumenti

WEKA

Lustre

BeeGFS

GPFS (IBM Spectrum Scale)

Pure Storage

NVMe-oF

NFS over RDMA

S3-compatible object storage

NVIDIA GPUDirect Storage

Descrizione del lavoro

We have a (12-Month Contract - Permanent) Domain Architect – AI Storage role available.

You will design, deploy and optimise high-performance storage infrastructure for large-scale AI factories, GPU clusters and High-Performance Computing (HPC) environments.

This is a hands-on architecture and delivery role focused on HPC storage, GPU data platforms, parallel file systems and high-throughput storage fabrics.

You will lead the storage component of NVIDIA SuperPOD, BasePOD and Cisco AI Factory deployments, ensuring storage infrastructure can feed large GPU clusters at scale and support demanding AI training, inference and enterprise workloads.

You will define repeatable storage architectures and technical standards for NVIDIA Cloud Provider (NCP) and private enterprise AI cloud environments, working closely with compute and network architects to deliver fully integrated AI infrastructure.

Key responsibilities

• Design, deploy and optimise HPC and AI storage infrastructure for GPU clusters and large-scale AI factories.
• Lead implementation of high-performance parallel file systems and storage platforms, including WEKA, DDN/Lustre, VAST Data, BeeGFS, GPFS and Pure Storage.
• Architect high-throughput storage fabrics capable of supporting AI training, inference, checkpointing and large-scale dataset processing.
• Implement and optimise NVMe-over-Fabrics (NVMe-oF), NVIDIA GPUDirect Storage (GDS), NFS over RDMA and S3-compatible object storage.
• Integrate storage with bare-metal GPU servers, Linux compute nodes, Kubernetes and Red Hat OpenShift.
• Configure Container Storage Interface (CSI) drivers, storage classes and persistent storage for AI workloads.
• Benchmark and troubleshoot storage performance using tools such as IOR, FIO and mdtest, identifying I/O bottlenecks that impact GPU utilisation.
• Develop automated deployment and configuration processes using Python, Ansible or Terraform.
• Design data lifecycle and tiering strategies across high-performance NVMe, flash and object storage.
• Collaborate with compute and network architects to ensure end-to-end performance across the Compute–Network–Storage architecture.
• Lead technical implementation, performance tuning, testing and validation of storage solutions.

Essential skills and experience

• Extensive experience designing and delivering high-performance storage solutions across HPC, AI, GPU or large-scale enterprise environments.
• Strong hands-on knowledge of parallel file systems such as WEKA, Lustre, BeeGFS or GPFS.
• Experience architecting storage for GPU clusters, AI training and/or inference workloads.
• Deep understanding of storage performance, throughput, IOPS, metadata performance and Linux I/O optimisation.
• Knowledge of NVMe-oF, RDMA-based storage and high-speed data access architectures.
• Experience with object storage and S3-compatible protocols for datasets, model checkpoints and archival workloads.
• Strong Linux systems knowledge, including file systems, block devices, client configuration and performance troubleshooting.
• Automation experience using Python, Ansible or Terraform.
• Ability to design scalable storage architectures and translate technical requirements into reliable, production-ready deployments.

Desirable experience

• Experience deploying storage for NVIDIA SuperPOD, BasePOD, DGX/HGX GPU clusters or Cisco AI Factory environments.
• Hands-on experience with NVIDIA GPUDirect Storage (GDS).
• Integration with Kubernetes, Red Hat OpenShift and CSI storage drivers.
• Understanding of InfiniBand and RoCEv2 networking from a storage performance perspective.