GPU AI Infra Reliability Engineer

Together AI

San Francisco (CA)

On-site

USD 250,000 - 320,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Startup equity
Health insurance
Competitive benefits

Job summary

Together AI, the AI Native Cloud, seeks a software engineer focused on reliability to own production outcomes for the Kubernetes-native control plane that provisions and runs a GPU inference fleet. You will triage InfiniBand issues, diagnose hero-model faults, and extend self-healing that scales with fleet size.

You will own SLOs, lead incident response, and drive fixes into controllers and automation while improving capacity efficiency and avoiding new instability.

Qualifications

  • Experience operating production reliability for large-scale distributed systems or GPU-serving infrastructures.
  • Hands-on troubleshooting InfiniBand/RDMA networking in multi-node contexts.
  • Familiarity with Kubernetes controller/CRD patterns and node-level health/remediation systems.

Responsibilities

  • Operate the provisioning state machine in production: health of host lifecycle, discovery, inference bring-up, GPU driver/CUDA validation, health checks, decommission/RMA, and triage of stuck transitions.
  • Extend automated self-healing: diagnose new failure modes live and codify into detection/remediation plugins for automatic rem remediation.
  • Own end-to-end reliability of the pipeline: hold SLOs and error budgets, lead incident responses, drive systemic fixes beyond hotfixes.
  • Partner with inference/ML platform teams to reflect real cluster topologies and constraints into platform abstractions.
  • Collaborate with ML researchers, engineers, and PMs to bring latest tech into production.
  • Introduce new techniques and tools to improve performance, latency, throughput, and efficiency of model inference.
  • Lead reliability across infra, ML platform, and product teams, shaping decisions before they ship.

Job description

Together AI, the AI Native Cloud, seeks a software engineer focused on reliability to own production outcomes for the Kubernetes-native control plane that provisions and runs a GPU inference fleet. You will triage InfiniBand issues, diagnose hero-model faults, and extend self-healing that scales with fleet size.

You will own SLOs, lead incident response, and drive fixes into controllers and automation while improving capacity efficiency and avoiding new instability.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Production GPU Inference Reliability Engineer
Production GPU Inference Reliability Engineer

FDE Times • San Francisco (CA)

On-site
USD 250,000 - 320,000
Startup equity
Health insurance
Competitive benefits
Senior AI Inference Reliability Engineer - GPU Compute Infra
Senior AI Inference Reliability Engineer - GPU Compute Infra

Together Computer Inc • San Francisco (CA)

On-site
USD 250,000 - 320,000
Equity
Health insurance
Competitive benefits
Senior AI Inference Reliability Engineer - GPU & Kubernetes
Senior AI Inference Reliability Engineer - GPU & Kubernetes

FDE Times • San Francisco (CA)

On-site
USD 160,000 - 280,000
Equity
Health insurance
Remote GPU Inference Infra Platform Engineer
Remote GPU Inference Infra Platform Engineer

Together Computer Inc • United States

Remote
USD 180,000 - 240,000
GPU Fleet Automation Engineer (Equity)
GPU Fleet Automation Engineer (Equity)

Together Computer Inc • San Francisco (CA)

On-site
USD 190,000 - 270,000
Health insurance
Equity
Competitive benefits
Cloud Infrastructure Engineer — GPU AI Kubernetes
Cloud Infrastructure Engineer — GPU AI Kubernetes

FriendliAI Inc. • San Francisco (CA)

On-site
USD 210,000 - 270,000
Flexible hours
Meals provided daily
Collaborative culture
+2
Senior Kubernetes Inference Platform Engineer
Senior Kubernetes Inference Platform Engineer

FDE Times • San Francisco (CA)

On-site
USD 160,000 - 280,000
Startup equity
Health insurance
Equity
Kubernetes Platform Engineer for AI Inference
Kubernetes Platform Engineer for AI Inference

present of AI • San Francisco (CA)

On-site
USD 180,000 - 240,000
Senior AI Infra Engineer: GPU Training & Automation
Senior AI Infra Engineer: GPU Training & Automation

Anduril Industries, Inc. • Seattle (WA)

On-site
USD 166,000 - 220,000
Autonomous GPU Infrastructure Engineer
Autonomous GPU Infrastructure Engineer

Sptix AI LLP • San Francisco (CA)

On-site
USD 190,000 - 270,000
Health insurance
Startup equity
Competitive benefits