Senior Staff Infra Engineer, AI Inference Platform

Togetherai; Amsterdam; On-site; EUR 110,000 - 160,000; Full time

Job summary

Together AI, the AI Native Cloud, is building an infrastructure-as-software platform to transform hardware lifecycles into scalable AI clusters. This role develops the state machines, declarative control planes, and robust workflows that manage from bare metal to production-grade inference clusters.

You will own software that provisions, scales, and heals GPU-backed infrastructure, with a product mindset and production-grade CI/CD processes.

Qualifications

• Strong software engineering background in Go, Python, Rust, or similar.
• Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent.
• Experience building software control planes or orchestration systems that model state and reconcile it over time.
• Experience with event-driven systems using message queues or pub/sub (Kafka, NATS, SQS).
• A product mindset with experience delivering internal platforms or APIs.

Responsibilities

• Build the provisioning state machine for full lifecycle of hardware, from discovery to decommission.
• Design a self-service API so the inference team can manage clusters with a single API call.
• Automate self-healing: detect degraded nodes, trigger repair, and reintroduce capacity.
• Own reliability of the pipeline with idempotency, retries, rollback, and drift detection.
• Collaborate with the inference/ML platform team to encode cluster shapes as abstractions.
• Engineer with strong typing, automated tests, code review, versioning, and CI/CD for infra code.

Skills

Go

Python

Rust

Temporal

Cadence

Kubernetes

Kafka

NATS

SQS

Product mindset

Tools

Temporal

Cadence

Kubernetes

Job description

Together AI, the AI Native Cloud, is building an infrastructure-as-software platform to transform hardware lifecycles into scalable AI clusters. This role develops the state machines, declarative control planes, and robust workflows that manage from bare metal to production-grade inference clusters.

You will own software that provisions, scales, and heals GPU-backed infrastructure, with a product mindset and production-grade CI/CD processes.