Staff Software Engineer, Kubernetes-native GPU Inference

Together AI; Greater London; Hybrid; GBP 100,000 - 160,000; Full time

Job summary

Together AI is seeking a Staff Software Engineer to design a Kubernetes-native control plane that provisions and runs a GPU inference fleet across London and Amsterdam. You’ll build a manifest-driven API, contribute self-service tooling, and optimize scheduling, defragmentation, and capacity balancing to push utilization while preserving latency.

You’ll own the provisioning lifecycle, build reliable pipelines, and partner with the ML platform team to encode cluster shapes as first-class

Qualifications

• Strong software engineering background with Go, Python, Rust, or similar.
• Experience with durable workflow orchestration tools (Temporal, Cadence, etc.).
• Experience building software control planes or orchestration systems (Kubernetes controllers/CRDs).
• Experience with event-driven systems using message queues or streams.

Responsibilities

• Build the provisioning state machine for full lifecycle of hosts, from discovery to decommission.
• Create a self-service API to allow the inference team to manage clusters without tickets.
• Automate self-healing: detect degraded nodes and trigger repairs or replacements.
• Own reliability of the pipeline with idempotency, retries, and drift detection.

Skills

Go

Python

Rust

Kubernetes controllers

Event-driven

GPU scheduling

Tools

Temporal

Cadence

Job description

Together AI is seeking a Staff Software Engineer to design a Kubernetes-native control plane that provisions and runs a GPU inference fleet across London and Amsterdam. You’ll build a manifest-driven API, contribute self-service tooling, and optimize scheduling, defragmentation, and capacity balancing to push utilization while preserving latency.

You’ll own the provisioning lifecycle, build reliable pipelines, and partner with the ML platform team to encode cluster shapes as first-class