Staff Software Engineer, AI Inference Infrastructure

Together AI; Amsterdam; On-site; EUR 90,000 - 130,000; Full time

Job summary

Together AI is seeking a Software Engineer to build the systems that treat infrastructure as software. You will design and implement state machines to provision hardware, bring it into service, and manage lifecycle, turning GPU racks into running AI clusters without manual runbooks.

You will write typed production code, tested and deployed via CI/CD, modeling infrastructure state and reconciling it automatically. You’ll own reliability and developer experience across the platform.

Qualifications

• Strong software engineering background in Go, Python, Rust, or similar.
• Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent.
• Experience building software control planes or orchestration systems that model state over time.
• Experience with event-driven systems (Kafka, NATS, SQS) rather than polling.
• A product mindset; built internal platforms or APIs used by other teams.

Responsibilities

• Build the provisioning state machine for the full lifecycle of a physical host from discovery to decommission.
• Develop the self-service API to stand up, scale, and tear down inference clusters with a single call.
• Automate self-healing: detect failures, drain, repair or replace, and reintroduce capacity.
• Own pipeline reliability: idempotency, retries, rollback, drift detection.
• Collaborate with ML platform teams on cluster shapes and constraints.
• Engineer with strong typing, tests, code reviews, and CI/CD for infrastructure code.

Skills

Go

Python

Rust

Distributed systems

Tools

Temporal

Cadence

Job description

Together AI is seeking a Software Engineer to build the systems that treat infrastructure as software. You will design and implement state machines to provision hardware, bring it into service, and manage lifecycle, turning GPU racks into running AI clusters without manual runbooks.

You will write typed production code, tested and deployed via CI/CD, modeling infrastructure state and reconciling it automatically. You’ll own reliability and developer experience across the platform.