Senior AI Infra Engineer: Inference Reliability & Scale

Together Computer Inc; San Francisco (CA); On-site; USD 250,000 - 320,000; Full time

Job summary

Together AI is hiring a Staff/Senior Staff AI Infra Engineer to own the production reliability of our GPU inference fleet. You will manage the Kubernetes-native control plane that provisions clusters, model deployments, and capacity changes, triage complex failures, and extend automated remediation across the fleet.

You will own SLOs, lead incident response, and translate postmortems into durable controller improvements, while ensuring high efficiency and stability of the inference stack in a

Qualifications

• Experience operating production reliability for large-scale distributed or GPU-serving systems.
• Hands-on troubleshooting InfiniBand/RDMA networking issues in multi-node contexts.
• Familiarity with multi-tenant model serving and SLOs across model pools.

Responsibilities

• Operate the provisioning/state machine in production: host lifecycle, health checks, and disaster recovery.
• Extend automated self-healing; add detection/remediation plugins for new failure modes.
• Own end-to-end reliability: SLOs, error budgets, incident response, and systemic fixes.

Skills

Kubernetes

InfiniBand/RDMA

GPU/ML serving

Production reliability

Incident response

Tools

Kubernetes CRD

Monitoring/Observability

Job description

Together AI is hiring a Staff/Senior Staff AI Infra Engineer to own the production reliability of our GPU inference fleet. You will manage the Kubernetes-native control plane that provisions clusters, model deployments, and capacity changes, triage complex failures, and extend automated remediation across the fleet.

You will own SLOs, lead incident response, and translate postmortems into durable controller improvements, while ensuring high efficiency and stability of the inference stack in a