Staff Software Engineer - AI Inference Platform & Provisioning

Togetherai; Amsterdam; On-site; EUR 90,000 - 120,000; Full time

Job summary

Together AI is seeking a Software Engineer to build systems that treat infrastructure as software, provisioning hardware and turning GPUs into running inference clusters. You will own lifecycle workflows, write typed production code, and operate it in production, aiming to minimize manual provisioning and maximize reliability.

You’ll design and implement the provisioning state machine, self-service API, and automated self-healing, collaborating with the ML platform team to meet cluster topology

Qualifications

• Strong software engineering background in Go, Python, or Rust.
• Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent.
• Experience building software control planes or orchestration systems that model state and reconcile it over time.
• Experience with event-driven systems (message queues, event streams, pub/sub).
• A product mindset and experience building internal platforms or APIs.

Responsibilities

• Build the provisioning state machine: model the full lifecycle of a physical host from discovery to decommission as explicit, versioned states.
• Build the self-service API: create declarative APIs to request, scale, and tear down clusters with one API call.
• Automate self-healing: detect degraded nodes, trigger repair or replacement, and maintain healthy capacity automatically.
• Own reliability of the pipeline: ensure idempotency, retries, rollback, and drift detection.
• Partner with the inference/ML platform team to encode cluster shapes as first-class abstractions.
• Engineer it like software: strong typing, automated tests, code review, versioning, and CI/CD for infrastructure code.

Skills

Go

Python

Rust

Event-driven design

Product mindset

Tools

Temporal

Cadence

Job description

Together AI is seeking a Software Engineer to build systems that treat infrastructure as software, provisioning hardware and turning GPUs into running inference clusters. You will own lifecycle workflows, write typed production code, and operate it in production, aiming to minimize manual provisioning and maximize reliability.

You’ll design and implement the provisioning state machine, self-service API, and automated self-healing, collaborating with the ML platform team to meet cluster topology