Senior Kubernetes Inference Platform Engineer

Fdetimes; San Francisco (CA); On-site; USD 160,000 - 280,000; Full time

Benefits offered by this job

Startup equity

Health insurance

Equity

Job summary

Together AI, the AI Native Cloud, is seeking a software engineer to build the Kubernetes-native control plane that provisions and runs our GPU inference fleet. You design a manifest-driven API where the inference team declares needs like clusters, model deployments, or capacity changes, and our controllers handle reconciliation, provider/runtime selection, and lifecycle management underneath.

The role focuses on decoupling operators from runtime complexity, building self-service surfaces, and

Qualifications

• Strong software engineering background in Go, Python, Rust, or similar — you write and test real software for a living.
• Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent to run long-lived, manifest-driven workflows that survive failures and resume mid-execution.
• Experience building software control planes or orchestration systems that model state and reconcile it over time (e.g., Kubernetes controllers/operators, custom reconciliation loops, workflow engines).
• Experience with event-driven systems — designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS) rather than polling or cron-driven scripts.
• A product mindset. You’ve built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship.

Responsibilities

• Build the provisioning state machine: model the full lifecycle of a physical host from discovery to GPU driver/CUDA stack, health validation, and decommission/RMA.
• Build the self-service API: declarative APIs and a control plane so the inference team can request, scale, and tear down clusters with one API call.
• Automate self-healing: detect degraded nodes, drain safely, trigger repair or replacement, and reintroduce healthy capacity.
• Own reliability of the pipeline: idempotency, retries, rollback, and drift detection for dependable provisioning.
• Partner with the inference/ML platform team to encode cluster shapes as first-class abstractions in the platform.
• Engineer with strong typing, automated tests, code review, versioning, and CI/CD for infrastructure code.

Skills

Go

Python

Rust

Temporal

Kubernetes

API design

Event-driven

Tools

Kubernetes

Temporal

Cadence

NATS

Kafka

Job description

Together AI, the AI Native Cloud, is seeking a software engineer to build the Kubernetes-native control plane that provisions and runs our GPU inference fleet. You design a manifest-driven API where the inference team declares needs like clusters, model deployments, or capacity changes, and our controllers handle reconciliation, provider/runtime selection, and lifecycle management underneath.

The role focuses on decoupling operators from runtime complexity, building self-service surfaces, and