About The Company
Our client is an AI-first enterprise planning software company on a mission to be the most value-creating platform for enterprises. Its platform integrates siloed planning capabilities and captures significant value leakage, helping global industry leaders plan smarter and faster and optimize their supply chains.
Location: Remote (US-based)
About The Company
Our client is an AI-first enterprise planning software company on a mission to be the most value-creating platform for enterprises. Its platform integrates siloed planning capabilities and captures significant value leakage, helping global industry leaders plan smarter and faster and optimize their supply chains.
About The Role
We are looking for an exceptional senior engineer to define and build the live-system reliability architecture for the company. The fundamental question for this role is simple: what is happening in production, what is about to go wrong, why, and what should we do?
You will design the control loop that reconstructs exact production state, assembles evidence across systems, detects both hard failures and weak signals, produces testable diagnoses, supports safe remediation, and learns from every incident. This is a hands‑on builder-architect role. You will set technical direction, write and review critical code, establish new abstractions and standards, and lead through technical influence across Reliability & Operations, Platform, Applications, AI, SecOps and Services/FDE.
Core technical thesis: AI should extend the reach of engineers, not replace engineering truth. Neural models can interpret ambiguous evidence and generate hypotheses; exact state, deterministic checks, policy, authorization and recovery determine what is safe to do.
What You’ll Do
- Own the architecture for the live-system reliability loop, including a precise operational model of environments, tenants, versions, configuration, extensions, integrations, data, infrastructure, workloads, deployments, ownership and time across company-hosted and customer-managed environments.
- Build evidence assembly across telemetry, traces, logs, alerts, deployments, configuration, tenant state, tickets, recent changes and code ownership, producing a coherent time‑ordered picture instead of disconnected artifacts.
- Drive detection and diagnosis across hard failures, weak signals, SLO and capacity risk, and recurring failure patterns. AI‑assisted root‑cause analysis should produce ranked, testable hypotheses with cited evidence, measured precision and clear abstention when evidence is insufficient.
- Create incident memory and versioned remediation skills with preconditions, permissions, blast‑radius limits, verification and rollback. Autonomy is earned only for failure classes that are well understood, measurable and reversible.
- Deliver high‑quality engineering escalations and a closed learning path from incidents into better detectors, runbooks, tests, capacity models, remediation paths and architecture.
- Make architecture decisions on central versus local or federated analysis, operational data boundaries, shared AI runtime versus team‑specific tools, and build‑versus‑buy choices.
What Success Looks Like
First 90 Days
- Learn the production estate deeply: hosted and customer‑managed deployment patterns, telemetry, support flows, incident classes, configuration variability, data boundaries and current reliability tooling and capabilities.
- Establish a common taxonomy and baseline for incidents, detection, diagnosis, recovery, recurrence, customer impact and engineering effort.
- Define the initial exact‑state model, topology model, evidence semantics, evaluation framework, action policy and audit requirements.
- Select one or two narrow, high‑value production pilots and produce an evidence‑backed architecture direction and two‑quarter delivery plan.
Over 12 to 24 Months
- Important recurring failure classes are anticipated or diagnosed automatically or with high‑quality AI assistance.
- Known reversible remedies become measured software capabilities rather than tribal runbooks.
- Production escalations consistently arrive with exact state, evidence and reproduction rather than requiring days of evidence gathering.
- Incident outcomes systematically improve future detection, diagnosis, remediation and change assurance.
- Human effort shifts toward prevention, architecture, novel failures and customer‑specific complexity while operational capacity scales more slowly than the customer estate.
What You’ll Bring
Education
- Bachelor's degree in Computer Science, Software Engineering, Information Technology, Industrial Engineering, Engineering Management, or a related field. Master's degree preferred.
Experience
- 15+ years of relevant experience.
- Evidence that you have personally designed, built and operated production reliability systems at meaningful scale in SRE, Production Engineering, distributed infrastructure, observability, cloud services or a closely related discipline.
- Deep software and distributed‑systems judgment. You can prototype, code, benchmark and debug the critical path yourself, not only review architecture diagrams.
- Strong understanding of telemetry, state reconstruction, distributed failure modes, incident response, SLOs, capacity, change correlation, reproducibility and safe remediation.
- Practical experience with modern AI systems: agents, tool use, retrieval, coding models, evaluation and their failure modes. You understand when probabilistic reasoning is useful and where deterministic validation is mandatory.
- A bias toward explicit models, typed state, evidence, invariants, authority and recovery rather than opaque automation.
- Judgment to adapt practices from large‑scale internet and cloud systems to a highly configurable enterprise environment: shared pre‑production, customer‑specific state, upgrades, and environments in which data may not be allowed to move.
- The ability to influence across organizational boundaries without relying on formal authority, and the humility to learn from the engineers and operators who know the current estate.
About The Culture
Transparency and open communication are core to the culture, and collaboration thrives across all levels. The company is among the fastest‑growing technology companies in its space, with a global presence across North America, Europe and Asia, and is an equal‑opportunity employer that welcomes applicants from all backgrounds.