Remote AI Infra Engineer — Scale GPU Clusters

Bright Vision Technologies; Monroeville (Allegheny County); Remote; USD 100,000 - 160,000; Full time

Job summary

Bright Vision Technologies is seeking an AI Infrastructure Engineer to design, build, and operate the platform layer powering large-scale AI training and inference workloads, with emphasis on GPU clusters, scheduling, storage, and developer experience for ML researchers.

The ideal candidate has built or operated production AI infrastructure at scale, understands hardware, kernel, scheduler, and ML framework interactions, and brings disciplined software engineering to platform work.

Qualifications

• Ten or more years of experience in infrastructure, platform, or HPC engineering.
• Hands-on experience operating GPU clusters or large-scale ML training infrastructure.
• Proficiency in Python and a systems language (Go or C++).
• Deep understanding of distributed training, accelerator architectures, and collective communication.
• Experience with Kubernetes, Slurm, Ray for ML workloads.

Responsibilities

• Design and operate GPU and accelerator infrastructure for training and inference, spanning on-prem clusters, cloud-managed services, and hybrid configurations.
• Build scheduling, queueing, and resource-sharing systems that maximize accelerator utilization across many teams.
• Integrate frameworks such as PyTorch, JAX, DeepSpeed, FSDP, Megatron-LM, and Ray Train into a unified platform offering.
• Operate high-performance storage systems and data pipelines that keep accelerators fed with training data at near-line-rate.
• Design networking architectures supporting RDMA, InfiniBand, NCCL, and high-bandwidth collective communication.
• Build observability for AI workloads including utilization, throughput, training stability, and failure-mode analytics.
• Implement checkpointing, restart, and fault-tolerance patterns for long-running training jobs at scale.
• Drive cost optimization across compute, storage, and networking through scheduling, spot capacity, and right-sizing.
• Develop developer tooling and paved-road workflows that let researchers launch experiments safely and efficiently.
• Partner with research and applied ML teams to plan capacity for upcoming training runs.
• Implement security controls, isolation, and access management for multi-tenant AI infrastructure.
• Drive automation across cluster provisioning, lifecycle management, and configuration enforcement.
• Maintain runbooks, capacity dashboards, and operational documentation for the AI platform.
• Stay current with AI infrastructure research, accelerator hardware, and emerging open-source AI tooling.

Skills

Python

Go

C++

Distributed training

GPU clusters

CI/CD

Linux

Networking

Cloud platforms

Education

Bachelor’s or Master’s degree in Computer Science

Tools

Kubernetes

Slurm

Ray

PyTorch

JAX

DeepSpeed

FSDP

Megatron-LM

Ray Train

Job description

Bright Vision Technologies is seeking an AI Infrastructure Engineer to design, build, and operate the platform layer powering large-scale AI training and inference workloads, with emphasis on GPU clusters, scheduling, storage, and developer experience for ML researchers.

The ideal candidate has built or operated production AI infrastructure at scale, understands hardware, kernel, scheduler, and ML framework interactions, and brings disciplined software engineering to platform work.