Staff AI Inference Infra Engineer - Self-Service Platform

Together Computer Inc; San Francisco (CA); On-site; USD 240,000 - 280,000; Full time

Benefits offered by this job

Health insurance

Equity

Competitive benefits

Job summary

Together AI is hiring a Software Engineer to build the systems that treat infrastructure as software for AI workloads. You will own provisioning state machines, a declarative control plane, and CI/CD pipelines to transform racks of GPUs into scalable inference clusters with minimal human touch.

You will work with the Research and Inference team, delivering a product-focused platform that operators can rely on for standup, scale, and tear-down of AI clusters in production.

Qualifications

• Strong software engineering background in Go, Python, Rust, or similar.
• Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent.
• Experience building software control planes or orchestration systems that model state and reconcile it over time.
• Experience with event‑driven systems — message queues, event streams, or pub/sub.
• A product mindset: internal platforms or APIs used by other teams.

Responsibilities

• Build the provisioning state machine: model full lifecycle of a physical host from discovery to decommission/RMA as explicit states.
• Build the self-service API: declarative APIs to request, scale, and tear down clusters via one API call.
• Automate self-healing: detect degraded nodes, drain, repair or replace, and reintroduce capacity.
• Own reliability of the pipeline: idempotency, retries, rollback, drift detection.
• Partner with the inference/ML platform team to encode cluster shapes as abstractions.
• Engineer it like software: typing, tests, versioning, CI/CD for infrastructure code.

Skills

Go

Python

Rust

Distributed systems

Event-driven design

Product mindset

Job description

Together AI is hiring a Software Engineer to build the systems that treat infrastructure as software for AI workloads. You will own provisioning state machines, a declarative control plane, and CI/CD pipelines to transform racks of GPUs into scalable inference clusters with minimal human touch.

You will work with the Research and Inference team, delivering a product-focused platform that operators can rely on for standup, scale, and tear-down of AI clusters in production.