AI Infrastructure Engineer - GPU Fleet Automation

Together AI

United States

Remote

USD 140,000 - 220,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Together AI, the AI Native Cloud, seeks engineers to build and operate a world-scale GPU fleet for frontier model training and inference. You’ll design systems to automate provisioning, validation, deployment, upgrades, and repairs of GPU clusters with minimal human intervention.

You’ll create AI infrastructure agents for deployment, fault diagnosis, and autonomous remediation, and develop fleet intelligence to monitor health, predict failures, and optimize performance across thousands of

Qualifications

  • 3+ years building distributed systems, infrastructure platforms, or large-scale backend software.
  • Strong software engineering skills in Python, Go, or Rust.
  • Experience building platforms, automation systems, or developer infrastructure.
  • Experience with Linux, Kubernetes, Terraform, Ansible, or similar infrastructure technologies.
  • Strong systems thinking across hardware and software.
  • Automation-first mindset to eliminate repeated tasks.
  • Bonus: GPU infrastructure, CUDA, NCCL, NVLink/NVSwitch, InfiniBand or RoCE networking.
  • Experience with bare-metal provisioning and lifecycle management.
  • Experience with large-scale AI training or inference clusters related to hardware health monitoring and prediction.

Responsibilities

  • Design and build fleet automation systems that provision, validate, deploy, upgrade, repair, and retire GPU clusters with minimal human intervention.
  • Build AI Infrastructure Agents that automate deployment, root-cause failures, incident triage, and autonomous remediation.
  • Develop Fleet Intelligence platforms that monitor hardware health, firmware, networking, storage, thermals, and workload performance to predict failures.
  • Build software that maximizes GPU availability, utilization, performance, and reliability across thousands of accelerators.
  • Create automated validation systems for GPUs, InfiniBand/RoCE fabrics, NVLink/NVSwitch, storage, and distributed AI workloads.
  • Build internal platforms and developer tools for software-driven infrastructure management.
  • Continuously improve deployment velocity, reliability, and operational efficiency through automation.
  • Partner with hardware, networking, platform, and AI teams to push AI infrastructure limits.

Skills

Distributed systems
Python
Go
Rust
Linux
Kubernetes
Terraform
Ansible
Automation

Tools

Terraform
Ansible
Kubernetes

Job description

Together AI, the AI Native Cloud, seeks engineers to build and operate a world-scale GPU fleet for frontier model training and inference. You’ll design systems to automate provisioning, validation, deployment, upgrades, and repairs of GPU clusters with minimal human intervention.

You’ll create AI infrastructure agents for deployment, fault diagnosis, and autonomous remediation, and develop fleet intelligence to monitor health, predict failures, and optimize performance across thousands of

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

GPU Fleet Automation Engineer (Equity)
GPU Fleet Automation Engineer (Equity)

Together Computer Inc • San Francisco (CA)

On-site
USD 190,000 - 270,000
Health insurance
Equity
Competitive benefits
Autonomous GPU Infrastructure Engineer
Autonomous GPU Infrastructure Engineer

Sptix AI LLP • San Francisco (CA)

On-site
USD 190,000 - 270,000
Health insurance
Startup equity
Competitive benefits
Principal AI Infrastructure Engineer — GPU Fleet
Principal AI Infrastructure Engineer — GPU Fleet

Nscale • Bellevue (WA)

On-site
USD 240,000 - 400,000
Medical, dental, vision
Flexible paid time off
Parental leave
+1
GPU Fleet Infra Engineer — Scale, Automation & Kubernetes
GPU Fleet Infra Engineer — Scale, Automation & Kubernetes

OpenAI, Inc. • New York (NY)

Hybrid
USD 180,000 - 240,000
Relocation assistance
Hybrid work model
GPU AI Infra Reliability Engineer
GPU AI Infra Reliability Engineer

Together AI • San Francisco (CA)

On-site
USD 250,000 - 320,000
Startup equity
Health insurance
Competitive benefits
AI Infrastructure System Engineer
AI Infrastructure System Engineer

Together AI • United States

Remote
USD 140,000 - 220,000
AI Infrastructure Engineer - GPU & Network
AI Infrastructure Engineer - GPU & Network

Sierra Business Solution LLC • San Jose (CA)

On-site
USD 140,000 - 210,000
AI Infrastructure Architect — Scalable GPU Compute
AI Infrastructure Architect — Scalable GPU Compute

EngineersOfAI • Sunnyvale (CA)

On-site
USD 150,000 - 200,000
AI Infra & Backend Engineer for Scalable GPU Cloud
AI Infra & Backend Engineer for Scalable GPU Cloud

GMI Cloud, Inc • Mountain View (CA)

On-site
USD 180,000 - 240,000
Robotics ML Infra Engineer: GPU Fleet & Inference
Robotics ML Infra Engineer: GPU Fleet & Inference

Generalist AI, Inc. • Boston (MA)

On-site
USD 150,000 - 230,000