Staff Engineer, Kubernetes GPU Inference Platform

Togetherai; Greater London; On-site; GBP 110,000 - 150,000; Full time

Job summary

Together AI, the AI Native Cloud, is seeking a software engineer to build a Kubernetes-native control plane that provisions and runs our GPU inference fleet.

You will design a manifest-driven API where the inference team declares what they need, whether that's a cluster, a model deployment, or a capacity change, and our controllers handle the reconciliation, provider/runtime selection, and lifecycle management underneath, so the inference team never has to know or care which specific serving

Qualifications

• Strong software engineering background and hands-on experience delivering production software.
• Experience with durable workflow orchestration tools to run long-lived, manifest-driven workflows.

Responsibilities

• Build the provisioning state machine: model full lifecycle of a host from discovery to GPU driver stack and decommission.
• Build the self-service API to let inference teams request, scale, and teardown clusters with one API call.
• Automate self-healing: detect degraded nodes, drain, repair or replace, and reintroduce capacity automatically.

Skills

Go

Python

Rust

Kubernetes controllers/operators

Event-driven systems

Product mindset

Temporal/Cadence

Tools

Temporal

Cadence

Job description

Together AI, the AI Native Cloud, is seeking a software engineer to build a Kubernetes-native control plane that provisions and runs our GPU inference fleet.

You will design a manifest-driven API where the inference team declares what they need, whether that's a cluster, a model deployment, or a capacity change, and our controllers handle the reconciliation, provider/runtime selection, and lifecycle management underneath, so the inference team never has to know or care which specific serving