Senior Site Reliability Engineer, Observability

blockchaincapital.com; New York (NY); On-site; USD 130,000 - 180,000; Full time

Job summary

Ripple is seeking a Senior Site Reliability Engineer to lead observability and incident management across production systems in a Windows/Azure environment. You will design dashboards in New Relic, implement SLOs/SLIs, tune alerts, and partner with engineering teams to improve reliability and performance.

Requirements include 7+ years in SRE/DevOps, hands-on engineering with mentoring, and strong experience with Terraform, Azure, AWS, and incident management platforms like Incident.IO or

Qualifications

• 7+ years in Site Reliability Engineering, DevOps, or Platform Engineering with a focus on observability and production operations.
• Proven ability to deliver hands-on engineering work while coaching and mentoring teams.
• Experience working in Agile/Scrum environments and collaborating with cross-functional teams.

Responsibilities

• Design and implement monitoring, alerting, and dashboards in New Relic across Azure and AWS; write NRQL queries for troubleshooting and reporting.
• Define and implement SLOs/SLIs and error budgets; coach teams on balancing velocity with reliability.
• Lead alert noise reduction, tune thresholds, and ensure actionable alerts.
• Optimize observability costs through log ingestion management and configuration governance.
• Partner with engineering to improve observability maturity: structured logging, metrics instrumentation, tracing, and dashboard patterns.
• Develop and maintain Terraform IaC for monitoring resources; enforce governance standards.
• Author and troubleshoot Azure DevOps pipelines; support deployment visibility and release hygiene.
• Administer Incident.IO: alert routing, Slack/OpsGenie integration, and runbooks; establish incident management foundations.
• Track MTTR/MTTD and incident frequency; drive continual improvement with engineering teams.
• Facilitate post-incident reviews and real-time troubleshooting with stakeholders.
• Enable teams through workshops and training; translate needs into self-service capabilities.

Skills

Site Reliability Engineering

DevOps

Observability

NRQL

Agile/Scrum

Mentoring

Cross-functional collaboration

Tools

Terraform

Azure

AWS

PowerShell

Azure DevOps

Octopus Deploy

Incident.IO

PagerDuty

OpsGenie

Slack

Jira

Job description

WHAT YOU'LL DO:

Observability Engineering

• Design and implement monitoring, alerting, and dashboards in New Relic (APM, Infrastructure, Logs, Synthetics) across Azure and AWS; write NRQL queries for troubleshooting, analysis, and reporting.
• Define and implement SLOs/SLIs and error budgets; coach teams on using them to balance feature velocity with reliability and communicate system health to stakeholders.
• Lead alert noise reduction and signal quality engineering—tune thresholds, eliminate false positives, and ensure every alert is actionable.
• Optimize observability costs through log ingestion management, pipeline rules, and New Relic configuration governance.
• Partner with engineering teams to improve observability maturity: structured logging, metrics instrumentation (RED/USE methods), distributed tracing, and effective dashboard patterns.

Infrastructure & IaC

• Develop and maintain Terraform infrastructure as code for provisioning and managing monitoring resources, alert configurations, and observability infrastructure—as a primary engineering responsibility.
• Establish and enforce IaC governance standards for observability infrastructure across teams, providing a repeatable, auditable model for how monitoring resources are managed.
• Author and troubleshoot Azure DevOps pipelines; support teams with deployment visibility, change tracking, and release hygiene as it relates to production reliability.

Incident Management

• Administer and configure Incident.IO: alert routing, notification workflows, Slack and OpsGenie integration, and runbook management—operationalizing what exists today and expanding from there.
• Build out incident management foundations that are largely yours to establish: PIR/postmortem processes, on‑call rotation design, escalation policies, incident severity classification, and response playbooks.
• Track and report on MTTR, MTTD, and incident frequency; identify trends and drive continuous improvement in partnership with engineering teams.
• Respond to and debrief on production incidents—providing real-time troubleshooting support and facilitating structured post‑incident reviews.

Cross-Functional Enablement

• Enable stream‑aligned engineering teams to adopt improved observability and incident management practices through workshops, consultation, and hands‑on guidance.
• Collaborate with the Subsystems Platform Team to translate common needs into self‑service observability and incident management capabilities.
• Build lasting team competency through documentation, training materials, and knowledge‑sharing sessions that outlast any individual engagement.

WHAT YOU'LL BRING:

Core SRE Experience

• 7+ years in Site Reliability Engineering, DevOps, or Platform Engineering with a strong focus on observability and production operations.
• Proven ability to deliver hands‑on engineering work while coaching and mentoring teams—comfortable switching between builder and consultant modes.
• Experience working in Agile/Scrum environments and collaborating effectively with cross‑functional teams.

Observability & Incident Management Expertise — Required

• Expert‑level hands‑on experience with New Relic (APM, Infrastructure, Logs, Synthetics, Alerts) and strong NRQL proficiency for troubleshooting and analysis.
• Deep understanding of structured logging, metrics collection (RED/USE methods), distributed tracing, and designing effective dashboards and alerts.
• Expertise defining and implementing SLOs/SLIs and error budgets for reliability management.
• Hands‑on experience with incident management platforms (Incident.IO, PagerDuty, OpsGenie, or similar).
• Experience designing incident response workflows, on‑call rotations, escalation policies, and facilitating post‑incident reviews that drive actionable improvements.
• Demonstrated ability to troubleshoot complex production issues using observability data across distributed systems.

Infrastructure & Tools — Required

• Strong Terraform experience: developing and maintaining IaC for cloud infrastructure and monitoring resources; familiarity with IaC governance patterns.
• Proficiency with PowerShell scripting (required given the 80% Windows environment).
• Strong experience with Azure cloud (App Services, Virtual Machines, Azure SQL, networking, monitoring) and working knowledge of AWS.
• Experience with Azure DevOps for CI/CD pipeline authoring and troubleshooting.
• Experience with Octopus Deploy for deployment management and release orchestration.
• Comfort working across both Windows and Linux server environments.
• Familiarity with Slack for operational workflows, alert routing, and incident communication.

Desired / Additional

• Experience with alert noise reduction strategies and observability cost optimization (log ingestion, pipeline rules, cardinality management).
• Background facilitating chaos engineering, game‑day exercises, or failure injection to build team resilience.
• Knowledge of VM‑hosted SQL Server monitoring and performance optimization.
• Familiarity with FinTech compliance requirements (SOC 2, ISO 27001) and audit evidence collection.
• Experience measuring and improving key reliability metrics (MTTR, MTTD, availability, error budgets) at an organizational level.
• Python or Bash scripting experience in addition to PowerShell.
• Familiarity with Jira for incident tracking and workflow automation.

Other common names for this role: Senior Site Reliability Engineer, Observability Engineer, Incident Management Engineer

Benefits listed above are for full‑time employees.

Ripple is an Equal Opportunity Employer. We’re committed to building a diverse and inclusive team. We do not discriminate against qualified employees or applicants because of race, color, religion, gender identity, sex, sexual identity, pregnancy, national origin, ancestry, citizenship, age, marital status, physical disability, mental disability, medical condition, military status, or any other characteristic protected by local law or ordinance.

Please find our UK/EU Applicant Privacy Notice and California Applicant Privacy Notice for reference.