Staff Infra Engineer, AI Inference & Compute Platform

Together; Amsterdam; On-site; EUR 110,000 - 150,000; Full time

Job summary

Together AI is seeking a Staff Software Engineer to build the systems that treat infrastructure as software, turning racks of GPUs into running inference clusters. You will design and implement stateful, manifest-driven engines and a declarative control plane, enabling scalable, self-service AI infrastructure for the research and inference teams.

You will own a production-grade platform with strong typing, automated tests, and CI/CD, collaborating closely with the ML platform team to meet

Qualifications

• Strong software engineering background in Go, Python, or Rust.
• Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent.
• Experience building software control planes or orchestration systems that model state and reconcile it over time.
• Experience with event-driven systems—message queues, event streams, or pub/sub.
• A product mindset; internal platforms or APIs consumed by other engineering teams.

Responsibilities

• Build the provisioning state machine: model full lifecycle of a host from discovery to decommission in explicit, versioned states and transitions.
• Build the self-service API and control plane so the inference team can manage clusters with one API call.
• Automate self-healing: detect degraded nodes, drain safely, trigger repair or replacement, and reintroduce capacity.
• Own reliability of the pipeline with idempotency, retries, rollback, and drift detection.
• Partner with the inference/ML platform team to encode cluster shapes as first-class abstractions in the platform.
• Engineer it like software: strong typing, automated tests, code review, versioning, and CI/CD for infrastructure code.

Skills

Go

Python

Rust

Testing

Tools

Temporal

Cadence

Job description

Together AI is seeking a Staff Software Engineer to build the systems that treat infrastructure as software, turning racks of GPUs into running inference clusters. You will design and implement stateful, manifest-driven engines and a declarative control plane, enabling scalable, self-service AI infrastructure for the research and inference teams.

You will own a production-grade platform with strong typing, automated tests, and CI/CD, collaborating closely with the ML platform team to meet