Kubernetes Platform Engineer for GPU Inference

Togetherai; San Francisco (CA); On-site; USD 160,000 - 280,000; Full time

Benefits offered by this job

Health insurance

Job summary

Together AI is hiring a software engineer to build a Kubernetes-native control plane that provisions and runs a GPU inference fleet. You’ll design a manifest-driven API and controllers that reconcile, select backends, and manage lifecycle, plus defragmentation and bin-packing to improve utilization.

You’ll own the platform end-to-end, including deployment, testing, and production support, with a focus on decoupling developers from underlying complexity.

Qualifications

• Strong software engineering background in Go, Python, Rust, or similar.
• Experience with durable workflow orchestration tools to run long-lived, manifest-driven workflows.
• Experience building software control planes or orchestration systems that model state and reconcile it over time.
• Experience with event-driven systems (message queues, event streams, pub/sub).
• A product mindset: built internal platforms or APIs consumed by other teams.

Responsibilities

• Build the provisioning state machine for full host lifecycle from discovery to decommission.
• Build a self-service API so the inference team can request and scale clusters with one API call.
• Automate self-healing: drain, repair or replace, reintroduce healthy capacity.
• Own reliability: idempotency, retries, rollback and drift detection.
• Partner with ML platform teams to encode cluster shapes as abstractions.
• Engineer with strong typing, tests, code review, CI/CD for infrastructure code.

Skills

Go

Python

Rust

Software engineering

Product mindset

Tools

Temporal

Cadence

Job description

Together AI is hiring a software engineer to build a Kubernetes-native control plane that provisions and runs a GPU inference fleet. You’ll design a manifest-driven API and controllers that reconcile, select backends, and manage lifecycle, plus defragmentation and bin-packing to improve utilization.

You’ll own the platform end-to-end, including deployment, testing, and production support, with a focus on decoupling developers from underlying complexity.