Senior SRE — GPU Fleet Orchestration & Autoscaling

Hippocratic-Ai; Menlo Park (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Hippocratic AI in Menlo Park, CA, seeks a Senior Site Reliability Engineer who writes production software and runs the infrastructure it lives on. You will own a GPU management and scheduling platform for ~30 models on heterogeneous hardware, shaping admission control, autoscaling, and reliability.

You'll mentor engineers, design scalable systems on AWS/GCP/Azure, implement Terraform pipelines and CI/CD, and ensure HIPAA-ready security and observability.

Qualifications

• 10+ years in SRE/DevOps and software engineering.
• CS degree from a top program is required.
• Strong Python/Go orchestration and scheduling experience.
• Experience building systems from operational metrics to control loops.
• Deep infra automation and CI/CD experience with Terraform or similar.
• Hands-on production cloud experience (AWS/GCP/Azure).

Responsibilities

• Design and build GPU management and scheduling platform for ~30 models.
• Build metrics pipeline and decision logic from GPU signals.
• Implement admission control to protect capacity and autoscaling.
• Develop cloud orchestration in Python/Go and automate deployments.
• Architect scalable, fault-tolerant production systems on cloud.
• Establish monitoring, logging, and alerting for reliability.
• Ensure HIPAA/compliance readiness and security best practices.
• Mentor engineers and raise technical bar across the team.

Skills

10+ years experience

Strong CS fundamentals

Scheduling systems in Python/Go

Autoscaling & load shedding

CI/CD & infra automation

Cloud platforms (AWS/GCP/Azure)

Containerization & orchestration

Monitoring & logging stacks

Security tooling / secrets management

Problem solving & communication

Education

Computer Science Degree

Tools

Terraform

GitLab CI/CD

Docker

Kubernetes

ELK

Grafana

Datadog

HashiCorp Vault

AWS

GCP

Azure

KMS

Job description

Hippocratic AI in Menlo Park, CA, seeks a Senior Site Reliability Engineer who writes production software and runs the infrastructure it lives on. You will own a GPU management and scheduling platform for ~30 models on heterogeneous hardware, shaping admission control, autoscaling, and reliability.

You'll mentor engineers, design scalable systems on AWS/GCP/Azure, implement Terraform pipelines and CI/CD, and ensure HIPAA-ready security and observability.