Senior AI Inference Reliability Engineer, GPU & Kubernetes

Teli Labs; San Francisco (CA); On-site; USD 160,000 - 280,000; Full time

Benefits offered by this job

Startup equity

Health insurance

Competitive benefits

Job summary

Together AI, the AI Native Cloud, seeks a Senior/Junior AI Inference Reliability Engineer to own production reliability for our Kubernetes-native control plane that provisions and runs GPU inference fleets in San Francisco, CA.

You’ll lead incident response, own SLOs and capacity efficiency, and extend automated remediation to handle new failure modes in multi-node GPU serving environments.

Qualifications

• Experience operating a provisioning or host-lifecycle pipeline in production.
• Deep experience owning production reliability for large-scale distributed systems.
• Hands-on troubleshooting InfiniBand/RDMA networking issues in multi-node environments.
• Familiarity with multi-tenant model serving and differentiated SLOs across model pools.
• Track record of automating operational toil through self-built tooling.
• Hands-on experience with Kubernetes controller/CRD patterns and health remediation.
• Software-engineering mindset with GPU infrastructure treated as software.

Responsibilities

• Operate the provisioning state machine in production and own host lifecycle health.
• Triage and extend automated self-healing; diagnose new failure modes and implement remediation.
• Own end-to-end reliability of the inference pipeline, SLOs, and incident response.
• Collaborate with inference/ML platform teams to reflect real cluster topologies and failures in design.
• Work with researchers, engineers, and product managers to productionize new tech.
• Introduce techniques and tools to improve performance, latency, throughput, and efficiency.
• Build visibility tools for bottlenecks and instability, and implement priority fixes.
• Lead reliability cross-functionally and influence roadmap decisions.

Job description

Together AI, the AI Native Cloud, seeks a Senior/Junior AI Inference Reliability Engineer to own production reliability for our Kubernetes-native control plane that provisions and runs GPU inference fleets in San Francisco, CA.

You’ll lead incident response, own SLOs and capacity efficiency, and extend automated remediation to handle new failure modes in multi-node GPU serving environments.