Senior SRE: GPU Fleet Orchestration & Auto-Scaling

Hippocratic AI

Menlo Park (CA)

On-site

USD 180,000 - 240,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Hippocratic AI is seeking a Senior Site Reliability Engineer to own the GPU management and scheduling platform that runs a fleet of ~30 models on heterogeneous hardware.

You will design metrics, admission control, and autoscaling, build infrastructure automation with Terraform and CI/CD, and operate secure production systems on AWS, GCP, or Azure.

Join a team building healthcare AI at scale, mentoring engineers and collaborating with researchers to ensure reliability, performance, and safety.

Qualifications

  • 10+ years of experience across site reliability, DevOps, and software engineering.
  • Computer Science degree from a top program.
  • Strong software engineering fundamentals with Python and/or Go for orchestration and scheduling.
  • Experience designing systems using operational metrics for autoscaling and admission control.
  • Deep experience with infrastructure automation and CI/CD (Terraform, GitLab CI/CD).
  • Hands-on production experience with AWS, GCP, or Azure.
  • Strong knowledge of Docker and Kubernetes.
  • Experience with monitoring/logging stacks (ELK, Grafana, Datadog).
  • Secrets management and security tooling (Vault, KMS, Key Vault).
  • Excellent problem-solving and communication skills.

Responsibilities

  • Design and build GPU management and scheduling platform across ~30 models on heterogeneous hardware.
  • Build metrics pipeline for GPU load and utilization and translate signals into decisions.
  • Implement admission control to protect capacity and manage inference requests.
  • Develop autoscaling for model replicas in real time.
  • Develop cloud orchestration in Python and Go to manage the model fleet.
  • Operate scalable, secure production systems on AWS, GCP, or Azure.
  • Create infrastructure automation and deployment pipelines (Terraform, CI/CD).
  • Establish monitoring, logging, and alerting to ensure reliability.
  • Enforce security/compliance for healthcare AI.
  • Collaborate with researchers to diagnose complex issues; mentor teammates.

Skills

SRE / DevOps
Python / Go
Cloud platforms AWS/GCP/AZ
Docker / Kubernetes
CI/CD tooling Terraform/GitLab
Monitoring & Logging
Security tooling Vault/KMS
Communication
Problem-solving
Autonomous collaboration

Education

Computer Science Degree from a top CS program

Tools

Terraform
GitLab CI/CD
Docker
Kubernetes
ELK
Grafana
Datadog
HashiCorp Vault
AWS KMS
Azure Key Vault

Job description

Hippocratic AI is seeking a Senior Site Reliability Engineer to own the GPU management and scheduling platform that runs a fleet of ~30 models on heterogeneous hardware.

You will design metrics, admission control, and autoscaling, build infrastructure automation with Terraform and CI/CD, and operate secure production systems on AWS, GCP, or Azure.

Join a team building healthcare AI at scale, mentoring engineers and collaborating with researchers to ensure reliability, performance, and safety.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Senior GPU Infrastructure SRE for AI Scale
Remote Senior GPU Infrastructure SRE for AI Scale

Andromeda Cluster • United States

Remote
USD 180,000 - 280,000
Senior SRE — GPU Infra, Auto-Healing & Automation
Senior SRE — GPU Infra, Auto-Healing & Automation

Salesonaut • Santa Clara (CA)

On-site
USD 168,000 - 270,000
Equity
Benefits package
Staff Site Reliability Engineer
Staff Site Reliability Engineer

Hippocratic AI • Menlo Park (CA)

On-site
USD 180,000 - 240,000
Senior SRE: Own AI/GPU Infra, Automation & Reliability
Senior SRE: Own AI/GPU Infra, Automation & Reliability

Nscale • San Francisco (CA)

On-site
USD 130,000 - 200,000
Equity
Career progression plan
Flexible schedule
Senior SRE - AI GPU Infra, OpenStack & Automation
Senior SRE - AI GPU Infra, OpenStack & Automation

Highrise AI • United States

Remote
USD 120,000 - 150,000
Staff Site Reliability Engineer
Staff Site Reliability Engineer

Hippocratic AI Inc. • Menlo Park (CA)

On-site
USD 180,000 - 240,000
Senior Global Capacity Manager - Remote AI GPU Fleet
Senior Global Capacity Manager - Remote AI GPU Fleet

present of AI • San Francisco (CA)

Remote
USD 190,000 - 260,000
Senior SRE - AI/GPU Cloud, Automation & Reliability
Senior SRE - AI/GPU Cloud, Automation & Reliability

Nscale • Seattle (WA)

On-site
USD 130,000 - 200,000
Equity
Career growth
Flexible work
Senior SRE — GPU Fleet Orchestration & Autoscaling
Senior SRE — GPU Fleet Orchestration & Autoscaling

Hippocratic AI Inc. • Menlo Park (CA)

On-site
USD 180,000 - 240,000
Senior SRE - Remote, GPU & FinOps Infra Lead
Senior SRE - Remote, GPU & FinOps Infra Lead

Level AI • Mountain View (CA)

Remote
USD 180,000 - 260,000