Senior AI Infrastructure Engineer — Global GPU Cloud

Together Computer Inc; United States; Remote; USD 160,000 - 230,000; Full time

Benefits offered by this job

Startup equity

Health insurance

Remote work flexibility

Job summary

Together AI is building a blazing-fast AI cloud platform and seeking a Senior AI Infrastructure Engineer to design, build, and operate backend services that manage cutting-edge hardware across global data centers. You will help create IaaS for thousands of GPUs, contribute to a high-performance object store, and craft observability and deployment tooling.

The role requires deep systems knowledge, strong software fundamentals, and excellent communication to collaborate across technical and

Qualifications

• 5+ years of professional software development experience and proficiency in at least one backend programming language (Golang desired)
• 5+ years experience writing high-performance, well-tested, production quality code
• Demonstrated experience with building and operating high-performance and/or globally distributed micro-service architectures across one or more cloud providers (AWS, Azure, GCP)
• Excellent communication skills – able to write clear design docs and work effectively with both technical and non-technical team members
• Deep experience with Kubernetes internals a big plus, such as implementing non-trivial Kubernetes operators, device/storage/network plugins, custom schedulers, or patches thereon or Kubernetes itself
• Deep experience with VMs/hypervisors a big plus, such as QEMU/KVM, cloud-hypervisor, VFIO, virtio, PCIE passthrough, Kubevirt, SR-IOV
• Deep experience with DC networking tech + solutions a big plus, such as VLAN, VXLAN, VPN, VPC, OVS/OVN
• Experience with Cluster API or similar a big plus
• Experience working on high-performance compute, networking, and/or storage a big plus
• Experience virtualizing GPUs and/or Infiniband a big plus
• Strong systems knowledge across compute, networking, and storage, including concurrency, memory management, performant I/O, and scale
• Experience with infrastructure automation tools (Terraform, Ansible), monitoring/observability stacks (Prometheus, Grafana), and CI/CD pipelines (GitHub Actions, ArgoCD)
• Experience building IaaS or PaaS systems at scale a plus
• Experience with DPUs/SmartNICs a plus
• GPU programming, NCCL, CUDA knowledge a plus

Responsibilities

• Design, build, and maintain performant, secure, and highly-available backend services/operators that run in our data centers and automate hardware management, such as Infiniband partitioning, in-DC parallel storage provisioning, and VM provisioning
• Design and build out the IaaS software layer for a new GB200 data center with thousands of GPUs
• Work on a global multi-exabyte high-performance object store, serving massive datasets for pretraining
• Build advanced observability stacks for our customers with automated node lifecycle management for fault-tolerant distributed pretraining
• Perform architecture and research work for decentralized AI workloads
• Work on the core, open-source Together AI platform
• Create services, tools, and developer documentation
• Create testing frameworks for robustness and fault-tolerance

Skills

Golang

Kubernetes internals

Distributed systems

Networking

CI/CD pipelines

Tools

Terraform

Ansible

Prometheus

Grafana

GitHub Actions

ArgoCD

Job description

Together AI is building a blazing-fast AI cloud platform and seeking a Senior AI Infrastructure Engineer to design, build, and operate backend services that manage cutting-edge hardware across global data centers. You will help create IaaS for thousands of GPUs, contribute to a high-performance object store, and craft observability and deployment tooling.

The role requires deep systems knowledge, strong software fundamentals, and excellent communication to collaborate across technical and