Senior Staff Software Engineer (Cloud Availability Platform)

Crusoe Energy Systems; Seattle (WA); On-site; USD 180,000 - 280,000; Full time

Job description

• Crusoe operates one of the world’s largest managed GPU fleets, and it is growing fast. A fleet at this scale cannot be run the way GPU clouds have traditionally been run: runbooks, war rooms, and heroics. It has to be run by a unified platform that senses, reasons about, and acts on the entire fleet, so infrastructure that used to take a team to operate takes a service instead
• That is what our cloud platform team builds. You will work on the control plane for one of the largest AI fleets in the world, at a point where fleet autonomy is still an open problem: nobody has fully solved this at this scale, in this market
• A true platform, not internal tooling: We want to be explicit about that heading, because it is the thing most infrastructure roles get wrong. Everything we build ships as a product, fleet engineers, SREs, and product teams across Crusoe build their own services and workflows on top of what we ship. A platform team does not scale by doing everyone’s work; it scales by making everyone’s work self-serve. Concretely:
• API-first. Every capability is exposed through well-designed, versioned APIs behind a single gateway. If it isn’t an API, it doesn’t exist. No side doors, including for us
• SDKs and paved paths. First-class client libraries, workflow templates, and golden paths so a fleet or SRE engineer can ship a new remediation or lifecycle workflow in days without asking the platform team
• Micro frontends and a self-serve portal. Teams plug their own UI surfaces into one developer portal instead of building one-off dashboards. One console for the fleet, extensible by every team
• Platform as product. Internal teams are customers. We own contracts, versioning, deprecation policy, quotas, documentation, and support. Adoption is our success metric: the platform wins when other teams choose it because it is the fastest path, not because it is mandated
• Four layers, built as one system:
• Agents on every site and host that collect telemetry and execute commands
• A distributed infra graph: Models system connections down to the rack, fabric, power, and cooling layers. By integrating these connections with telemetry signals, the platform can precisely trace events to identify their blast radius and root cause
• A reconciliation core: workflow engine, policy engine, and state reconciler that continuously close the gap between intended state and reality, exposed through the API gateway
• Domain services: Services spanning provisioning, firmware upgrade, validation, deployment, repair and RMA, capacity, power and thermal, and Day‑2 operations. Built once, run fleet-wide, consumable by any team through APIs and SDKs
• We operate on a continuous autonomy loop—sense, correlate, reason, act, learn—incorporating guardrails that evolve from recommendation to full automation. We treat every recurring manual intervention as a signal to engineer the next automation
• What you’ll do:
• Design and build core platform services: RBAC, tenancy, the workflow engine, policy engine, and state reconciler that drive fleet actions safely at scale
• Design the public face of the platform: the API gateway, resource model, and versioned API contracts that fleet, SRE, and product teams build against
• Build SDKs, workflow templates, and golden paths that make the platform self-serve, plus the developer portal and micro frontend framework that let teams bring their own UI surfaces
• Build the inventory and topology graph as the fleet’s source of intended truth, and the pipelines that keep it honest against reality (metadata drift is one of our top verified incident root causes; you will kill it)
• Build site, GPU, and network agents and the event bus that moves fleet telemetry and commands reliably
• Deliver the platform roadmap: pilot site on the foundation layer, first site deployed entirely through the platform, zero‑downtime firmware upgrades, first fully automated RMA, then 100K+ GPUs on platform with MTTD under 60 seconds and MTTR under 30 minutes
• Work with embedded engineers from fleet and production engineering who bring the operational scar tissue, and turn it into services other teams extend

Benefits

• Health & wellbeing: Comprehensive health benefits designed to support your overall wellness
• Time away: Paid time off for vacations, family bonding, and unexpected needs
• 401(k) match: Build your financial future with our 401(k) matching program
• Mental wellness: Resources and support for your emotional wellbeing and navigating life’s challenges

Measure yourself by what stops paging humans, and by how fast another team ships on your platformBelieve the interesting problems are where physical infrastructure meets software: a firmware counter, a thermal event, and a scheduling decision are one problem, not threeWant to build a platform, not integrate one. This is core distributed‑systems engineering: event buses, graph models, reconciliation loops, policy evaluationTreat internal engineers as customers and sweat API ergonomics, docs, and onboarding the way product teams sweat UXLike owning a hard abstraction and defending it as ten teams build on top of you10+ years building distributed systems, control planes, or infrastructure platformsDepth in at least one of: workflow/orchestration engines (Temporal or similar), event-driven architectures, graph data models, policy/rules engines, or reconciliation-based control loops (Kubernetes operator patterns)Bachelor’s degree in Computer Science, Data Science, or a closely related technical fieldExperience building platforms other engineers consume: public or internal APIs, SDKs, or developer tooling with real adoptionExperience running what you build: you have carried a pager for a platform other teams depend onStrong software engineering skills in Go, Python, or RustSystems thinking across the hardware/software boundaryInternal developer platforms: API gateways, service catalogs, Backstage‑style portals, micro frontend architecturesHigh-cardinality observability platforms (per‑GPU telemetry at fleet scale)GPU or bare-metal fleet infrastructure: DCGM, Redfish/IPMI, firmware lifecycleInfiniBand or RoCE fabricsAI agents applied to infrastructure triage and autonomous remediation