Senior AI Inference Reliability Engineer - GPU Infra

Together Computer Inc; San Francisco (CA); Sur place; USD 160 000 - 280 000; Plein temps

Avantages offerts par ce poste

Startup equity

Health insurance

Competitive benefits

Résumé du poste

Together AI is seeking a reliability-minded software engineer to own production reliability for its GPU inference fleet on a Kubernetes-based control plane. You will triage InfiniBand issues, extend auto-remediation, and drive SLOs and incident response across large-scale distributed systems.

Strong candidates have hands-on GPU infrastructure experience, familiarity with InfiniBand/RDMA, and a software-engineering mindset to treat infrastructure as code.

Qualifications

• Production reliability experience for large-scale distributed systems.
• GPU/ML serving infra experience is a strong plus.
• Hands-on troubleshooting InfiniBand/RDMA networking issues.
• Familiarity with multi-tenant model serving (LoRA/adapter-based serving).
• Automation and tooling you built yourself to reduce operational toil.
• Kubernetes controller/CRD patterns and reconciliation loops.

Responsabilités

• Operate the provisioning state machine in production: health of host lifecycle pipeline, discovery, inference bring-up, GPU driver/CUDA validation, health checks, decommission/RMA, triaging faults at fleet scale.
• Triage and extend automated self-healing: add new failure modes and deploy detection/remediation plugins.
• Own end-to-end reliability: hold SLOs and error budgets; lead incident response and drive systemic fixes.
• Collaborate with inference/ML platform to reflect real cluster topologies and scheduling constraints in platform abstractions.
• Work with researchers, engineers, and PMs to bring latest tech into production; enable advanced research via engineering.
• Introduce new techniques and architecture to improve performance, latency, throughput, and efficiency of the model inference stack.
• Build visibility tools to identify bottlenecks and instability; implement solutions for high-priority issues.
• Lead reliability cross-functionally across infra, ML platform, and product teams; influence planning with reliability priorities.

Connaissances

Production reliability

Kubernetes CRD

Observability

Automation/ tooling

GPU infrastructure

RDMA/InfiniBand

SLOs/incident response

Outils

Kubernetes

RDMA debugging tools

Description du poste

Together AI is seeking a reliability-minded software engineer to own production reliability for its GPU inference fleet on a Kubernetes-based control plane. You will triage InfiniBand issues, extend auto-remediation, and drive SLOs and incident response across large-scale distributed systems.

Strong candidates have hands-on GPU infrastructure experience, familiarity with InfiniBand/RDMA, and a software-engineering mindset to treat infrastructure as code.