Platform Engineer - GPU Infra & Kubernetes

Together AI; San Francisco (CA); On-site; USD 160,000 - 280,000; Full time

Benefits offered by this job

Equity

Health insurance

Competitive benefits

Job summary

Together AI, the AI Native Cloud, is hiring a software engineer to build a Kubernetes-native control plane for provisioning and running a GPU inference fleet. You will design a manifest-driven API where inference teams request clusters or models, with controllers handling reconciliation, provider selection, and lifecycle management.

The role emphasizes decoupling users from runtime complexity, improving utilization via defragmentation, bin-packing, and right-sizing, and owning the end-to-end

Qualifications

• Strong software engineering background with real software delivery experience.
• Experience building control planes or orchestration systems that model state over time.
• Experience with event-driven messaging and modern queues or pub/sub systems.
• Product-minded developer who ships usable platform APIs for other teams.

Responsibilities

• Build provisioning state machine covering discovery to decommission and repair.
• Create self-service API and control plane for request, scale, tear down.
• Automate self-healing, draining, and capacity reintroduction.
• Ensure reliability with idempotency, retries, and drift detection.
• Collaborate with ML/inference teams to encode cluster requirements as abstractions.
• Develop software-like practices with strong typing, tests, and CI/CD.

Skills

Go/Python/Rust

Kubernetes controllers

Event-driven systems

Product mindset

Testing & CI/CD

Tools

Temporal/Cadence

Kafka/NATS/SQS

Kubernetes

Job description

Together AI, the AI Native Cloud, is hiring a software engineer to build a Kubernetes-native control plane for provisioning and running a GPU inference fleet. You will design a manifest-driven API where inference teams request clusters or models, with controllers handling reconciliation, provider selection, and lifecycle management.

The role emphasizes decoupling users from runtime complexity, improving utilization via defragmentation, bin-packing, and right-sizing, and owning the end-to-end