Principal SRE — AI Infra Architecture

Nscale

New York, San Francisco, Seattle, Houston (NY, CA, WA, TX)

Remote

USD 150,000 - 250,000

Full time

9 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Competitive base + equity

Job summary

Nscale is seeking a Principal Site Reliability Engineer, Infrastructure Architecture to be the architectural authority for how our AI infrastructure is designed, connected, and recovered. You will own reference architectures for GPU, network, and control-plane platforms, define failure domains and trust boundaries, and ensure all systems have blast radii, recovery paths, and clear owners.

This role sits at the intersection of design and operations.

Qualifications

  • 10+ years building large-scale production infrastructure.
  • Track record of architectures designed and operating in production.
  • Led major incidents and large migrations or site bring-ups.
  • Knowledge of change management, staged rollouts, and SLO-based alerting.
  • Deep expertise in distributed systems design, consistency and replication.

Responsibilities

  • Own the end-to-end reference architecture for AI/HPC infrastructure: compute, network, storage, control plane and core services.
  • Design multi-site/multi-region topologies with failure domains and secure enclaves for sovereign/hyperscale customers.
  • Separate control plane from data plane to ensure management outages don’t affect workloads.
  • Define target-state architecture and multi-year roadmaps; decide what to retire or rebuild.
  • Lead architecture reviews and write ADRs/RFCs used by engineering teams.
  • Set standards for reliability: SLOs, observability, capacity models, dependency maps, failure modes.
  • Run pre-mortems and red-team reviews to uncover single points of failure.
  • Join major incidents when architecture is root cause and drive fixes back into design.
  • Co-own operating model: ORR, PIR, production access, break-glass policy.
  • Ensure teams can deploy, upgrade, and recover systems without constant hand-holding.
  • Plan site bring-ups, fleet upgrades, and migrations of live workloads.
  • Run DR tests and game days to validate recovery.

Skills

Distributed systems
Architectural leadership
SRE/DevOps
Incident management
Mentoring engineers
Technical writing

Education

BS in Computer Science or related field

Tools

Kubernetes
Terraform
Cloud platforms

Job description

Nscale is seeking a Principal Site Reliability Engineer, Infrastructure Architecture to be the architectural authority for how our AI infrastructure is designed, connected, and recovered. You will own reference architectures for GPU, network, and control-plane platforms, define failure domains and trust boundaries, and ensure all systems have blast radii, recovery paths, and clear owners.

This role sits at the intersection of design and operations.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior SRE: Own AI/GPU Infra, Automation & Reliability
Senior SRE: Own AI/GPU Infra, Automation & Reliability

Nscale • San Francisco (CA)

On-site
USD 130,000 - 200,000
Equity
Career progression plan
Flexible schedule
Principal SRE: AI-Driven Infra & Cloud Reliability
Principal SRE: AI-Driven Infra & Cloud Reliability

Candidate • Santa Clara (CA)

Hybrid
USD 212,000 - 265,000
Health plans
Parental leave
Retirement options
+2
Principal Infra Engineer — AI & HPC Scale
Principal Infra Engineer — AI & HPC Scale

nscaleoperationsukltd • United States

Remote
USD 180,000 - 240,000
Principal Site Reliability Engineer - AI Security Infra
Principal Site Reliability Engineer - AI Security Infra

Palo Alto Networks, Inc. • Santa Clara (CA)

On-site
USD 156,000 - 253,000
Lead AI Platform Reliability Architect
Lead AI Platform Reliability Architect

NVIDIA Corporation • Santa Clara (CA)

Hybrid
USD 248,000 - 397,000
Principal SRE: AI-Driven Cloud Reliability Leader
Principal SRE: AI-Driven Cloud Reliability Leader

Zscaler, Inc. • United States

Hybrid
USD 212,000 - 265,000
Principal AI Platform Architect & Reliability Lead
Principal AI Platform Architect & Reliability Lead

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 272,000 - 431,000
Equity
Benefits
Senior SRE - AI/GPU Cloud, Automation & Reliability
Senior SRE - AI/GPU Cloud, Automation & Reliability

Nscale • Seattle (WA)

On-site
USD 130,000 - 200,000
Equity
Career growth
Flexible work
Principal SRE - AI-Powered Cloud Reliability Leader
Principal SRE - AI-Powered Cloud Reliability Leader

Zscaler • Santa Clara (CA)

Hybrid
USD 212,000 - 265,000
Principal AI Infrastructure Network Engineer (Equity)
Principal AI Infrastructure Network Engineer (Equity)

Nscale • Houston (TX), New York (NY), San Francisco (CA), Seattle (WA)

On-site
USD 180,000 - 240,000
Base salary + equity
Equity incentives
Dynamic progression plan