Senior AI Infra Engineer — Global GPU Cloud

Together AI; Amsterdam; Hybrid; EUR 100,000 - 150,000; Full time

Job summary

Together AI is building the AI Acceleration Cloud, a scalable platform for the full generative AI lifecycle. As a Senior AI Infrastructure Engineer, you will help design, build, and operate a highly available cloud infra that virtualizes cutting-edge hardware and enables self-serve ML services for internal and external customers.

The role combines deep systems engineering with collaboration across teams. Hybrid work is available, with two days per week in the Amsterdam office, and a focus on

Qualifications

• 5+ years of professional software development experience.
• Proficiency in at least one backend language (Golang desired).
• Experience building and operating high-performance, globally distributed micro-service architectures across one or more cloud providers (AWS, Azure, GCP).
• Excellent communication skills with ability to write design docs for technical and non-technical stakeholders.
• Deep experience with Kubernetes internals and operators, plugins, or custom schedulers.
• Deep experience with VMs/hypervisors (QEMU/KVM, cloud-hypervisor, VFIO, virtio, KubeVirt, SR-IOV).
• Experience with DC networking (VLAN, VXLAN, VPN, VPC).
• Experience with Terraform/Ansible, monitoring stacks (Prometheus, Grafana), and CI/CD (GitHub Actions, ArgoCD).
• Experience building IaaS/PaaS at scale and GPU/DPUs a big plus.

Responsibilities

• Design, build, and maintain backend services/operators for data centers and hardware management automation.
• Design and build the IaaS layer for a GB200 data center with thousands of GPUs.
• Work on a global multi-exabyte high-performance object store for datasets.
• Build observability stacks with automated node lifecycle management for fault-tolerant pretraining.
• Architect and research decentralized AI workloads.
• Contribute to core Open Source Together AI platform.
• Create tools, developer documentation, and testing frameworks for robustness.

Skills

Golang

Distributed systems

Communication

Backend development

Tools

Terraform

Ansible

Prometheus

Grafana

GitHub Actions

ArgoCD

Kubernetes

Job description

Together AI is building the AI Acceleration Cloud, a scalable platform for the full generative AI lifecycle. As a Senior AI Infrastructure Engineer, you will help design, build, and operate a highly available cloud infra that virtualizes cutting-edge hardware and enables self-serve ML services for internal and external customers.

The role combines deep systems engineering with collaboration across teams. Hybrid work is available, with two days per week in the Amsterdam office, and a focus on