Senior AI Inference Reliability Engineer - GPU Compute Infra

Together Computer Inc

San Francisco (CA)

On-site

USD 250,000 - 320,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Equity
Health insurance
Competitive benefits

Job summary

Together AI, the AI Native Cloud, is seeking a Staff AI Inference Reliability Engineer to own production reliability for GPU inference fleets and the Kubernetes-native control plane. You will triage failures, manage SLOs, and drive automated remediation across a distributed, multi-node environment.

You will work with ML researchers, engineers, and product managers to ship reliable tooling, improve latency and throughput, and reduce operational toil through automation.

Qualifications

  • Experience with production reliability on large-scale distributed or GPU-serving systems.
  • Ability to diagnose InfiniBand/RDMA issues in multi-node environments.
  • Experience with Kubernetes controller/CRD architectures and reconciliation loops.
  • Willingness to engineer GPU infrastructure as software.

Responsibilities

  • Own health of the host lifecycle and fleet-scale provisioning workflows.
  • Lead incident response and translate postmortems into permanent fixes.
  • Extend automated self-healing and add new failure-detection plugins.
  • Collaborate with infra, ML platform, and product teams on reliability.
  • Build visibility tools to identify bottlenecks and instability.

Skills

Production reliability
Kubernetes
RDMA/InfiniBand
GPU infrastructure

Tools

Kubernetes CRD
InfiniBand
RDMA

Job description

Together AI, the AI Native Cloud, is seeking a Staff AI Inference Reliability Engineer to own production reliability for GPU inference fleets and the Kubernetes-native control plane. You will triage failures, manage SLOs, and drive automated remediation across a distributed, multi-node environment.

You will work with ML researchers, engineers, and product managers to ship reliable tooling, improve latency and throughput, and reduce operational toil through automation.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Inference Reliability Engineer - GPU & Kubernetes
Senior AI Inference Reliability Engineer - GPU & Kubernetes

FDE Times • San Francisco (CA)

On-site
USD 160,000 - 280,000
Equity
Health insurance
Production GPU Inference Reliability Engineer
Production GPU Inference Reliability Engineer

FDE Times • San Francisco (CA)

On-site
USD 250,000 - 320,000
Startup equity
Health insurance
Competitive benefits
Remote GPU Inference Infra Platform Engineer
Remote GPU Inference Infra Platform Engineer

Together Computer Inc • United States

Remote
USD 180,000 - 240,000
Senior Staff AI/ML Inference Engineer - GPU, Kubernetes
Senior Staff AI/ML Inference Engineer - GPU, Kubernetes

ServiceNow • Santa Clara (CA)

Hybrid
USD 201,000 - 352,000
Health plans
401(k) Plan with company match
ESP P (Employee Stock Purchase Plan)
+3
Senior AI Infra Engineer: GPU Training & Automation
Senior AI Infra Engineer: GPU Training & Automation

Anduril Industries, Inc. • Seattle (WA)

On-site
USD 166,000 - 220,000
AI/ML GPU Infra SRE — Build, Scale, Sustain
AI/ML GPU Infra SRE — Build, Scale, Sustain

GMI Cloud, Inc • United States

On-site
USD 120,000 - 170,000
Senior AI Infra Engineer - Equity Eligible GPU Cloud
Senior AI Infra Engineer - Equity Eligible GPU Cloud

NVIDIA Corporation • United States

Hybrid
USD 152,000 - 288,000
Equity
Benefits
Kubernetes Platform Engineer for AI Inference
Kubernetes Platform Engineer for AI Inference

present of AI • San Francisco (CA)

On-site
USD 180,000 - 240,000
Senior Kubernetes Inference Platform Engineer
Senior Kubernetes Inference Platform Engineer

FDE Times • San Francisco (CA)

On-site
USD 160,000 - 280,000
Startup equity
Health insurance
Equity
(Senior/Junior AI Inference Reliability Engineer, Compute Infra
(Senior/Junior AI Inference Reliability Engineer, Compute Infra

FDE Times • San Francisco (CA)

On-site
USD 160,000 - 280,000
Equity
Health insurance