Senior AI Inference Reliability Engineer, GPU/K8s

Teli Labs; San Francisco (CA); On-site; USD 250,000 - 320,000; Full time

Benefits offered by this job

Startup equity

Health insurance

Competitive benefits

Job summary

Together AI, the AI Native Cloud, seeks a Staff/ Senior Staff AI Inference Reliability Engineer to own production reliability for its GPU inference fleet. You will manage the Kubernetes-native control plane, extend self-healing, and ensure SLOs are met while driving automation and fault-tolerant design across multi-node GPU serving environments.

You’ll collaborate with ML researchers and product teams to push resilient inference at scale, improving latency, throughput and capacity efficiency

Qualifications

• Production reliability for large-scale systems, preferably GPU serving.
• Hands-on with Kubernetes controller/CRD patterns and tooling.
• Experience reducing toil via automation and self-healing systems.
• Strong software engineering mindset and production-grade coding.

Responsibilities

• Own and improve the host lifecycle pipeline reliability in production.
• Extend automated self-healing with new failure modes and plugins.
• Maintain SLOs and error budgets for provisioning and fleet health.
• Collaborate with ML platform teams to model real-world failures.
• Lead cross-functional reliability initiatives and incident response.
• Develop visibility tools to identify bottlenecks and instabilities.

Skills

Production reliability

Kubernetes

InfiniBand/RDMA

Distributed systems

Automation

Observability

Software engineering mindset

Tools

Kubernetes CRD patterns

Controller patterns

Monitoring tooling

Job description

Together AI, the AI Native Cloud, seeks a Staff/ Senior Staff AI Inference Reliability Engineer to own production reliability for its GPU inference fleet. You will manage the Kubernetes-native control plane, extend self-healing, and ensure SLOs are met while driving automation and fault-tolerant design across multi-node GPU serving environments.

You’ll collaborate with ML researchers and product teams to push resilient inference at scale, improving latency, throughput and capacity efficiency