Staff AI Infrastructure Orchestrator

Togetherai; Amsterdam; On-site; EUR 120,000 - 180,000; Full time

Job summary

Together AI is seeking a Software Engineer to build and operate an infra platform that turns racks of GPUs into running inference clusters. You will design and implement the provisioning state machine, self‑service API, and automated repair workflows to deliver fully automated lifecycle management from bare metal to AI clusters.

You will work on typed, tested software deployed via CI/CD, aligning with a product mindset to provide reliable infrastructure that scales with research and inference

Qualifications

• Strong software engineering background in Go, Python, Rust, or similar — you write and test real software for a living.
• Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent to run long‑lived, manifest‑driven workflows that survive failures and resume mid‑execution.
• Experience building software control planes or orchestration systems that model state and reconcile it over time (e.g., Kubernetes controllers/operators, custom reconciliation loops, workflow engines).
• Experience with event‑driven systems — designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS) rather than polling or cron‑driven scripts.
• A product mindset. You’ve built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship.

Responsibilities

• Build the provisioning state machine: design and implement the software that models the full lifecycle of a physical host from discovery, inference bring-up to GPU driver/CUDA stack, health validation, and decommission/RMA — as explicit, versioned states and transitions.
• Build the self-service API: design declarative APIs and a control plane so the inference team can request, scale, and tear down inference clusters with one API call — no ticket, no human in the loop.
• Automate self-healing: detect degraded or failed nodes, drain them safely, trigger repair or replacement, and reintroduce healthy capacity into the pool automatically.
• Own reliability of the pipeline: idempotency, retries, rollback, and drift detection so the provisioning system is as dependable as any other production service.
• Partner with the inference/ML platform team: understand the cluster shapes they need — topology, interconnect, scheduling constraints — and encode them as first‑class abstractions in the platform.
• Engineer it like software: strong typing, automated tests, code review, versioning, and CI/CD for infrastructure code — this is a product, not a collection of Ansible playbooks.

Skills

Go

Python

Rust

software design

CI/CD

Tools

Temporal

Cadence

Kubernetes

Job description

Together AI is seeking a Software Engineer to build and operate an infra platform that turns racks of GPUs into running inference clusters. You will design and implement the provisioning state machine, self‑service API, and automated repair workflows to deliver fully automated lifecycle management from bare metal to AI clusters.

You will work on typed, tested software deployed via CI/CD, aligning with a product mindset to provide reliable infrastructure that scales with research and inference