Senior AI Inference Reliability Engineer - GPU & Kubernetes

Fdetimes; San Francisco (CA); Vor Ort; USD 160.000 - 280.000; Vollzeit

Benefits dieser Stelle

Equity

Health insurance

Zusammenfassung

Together AI in the United States is seeking a software engineer focused on reliability to own production for the Kubernetes-native control plane powering GPU inference fleets. You’ll manage SLOs, incident response, and automated remediation while improving efficiency and resilience at large scale.

You will collaborate with ML platforms and researchers to bring cutting-edge inference tech into production, building robust tooling and architecture to minimize outages and maximize throughput.

Qualifikationen

• Experience operating a provisioning or host-lifecycle pipeline in production.
• Deep experience owning production reliability for large-scale distributed systems; GPU/ML-serving infra is a plus.
• Hands-on troubleshooting InfiniBand/RDMA networking issues in multi-node contexts.
• Familiarity with multi-tenant model serving and managing differentiated SLOs.
• Proven automation to eliminate operational toil with self-built tooling.

Aufgaben

• Operate the provisioning state machine in production and own host lifecycle health.
• Triage and extend automated self-healing, diagnosing new failure modes live.
• Own end-to-end reliability of the pipeline and drive systemic fixes.
• Partner with ML platform teams to reflect real cluster topologies and constraints.
• Work with researchers and engineers to deploy new tech into production.
• Introduce techniques and tools to improve performance, latency, throughput, efficiency.
• Build visibility tools and address bottlenecks and instability.

Kenntnisse

Kubernetes controllers/CRD

InfiniBand/RDMA troubleshooting

Distributed systems reliability

GPU inference infrastructure

Automation tooling

Observability (metrics/logs/traces)

Tools

RDMA networking tools

GPU driver validation

Jobbeschreibung

Together AI in the United States is seeking a software engineer focused on reliability to own production for the Kubernetes-native control plane powering GPU inference fleets. You’ll manage SLOs, incident response, and automated remediation while improving efficiency and resilience at large scale.

You will collaborate with ML platforms and researchers to bring cutting-edge inference tech into production, building robust tooling and architecture to minimize outages and maximize throughput.