Senior Observability & Reliability Engineer

U.S. Bank; Cupertino (CA); On-site; USD 124,000 - 146,000; Full time

Benefits offered by this job

Healthcare

Life insurance

Disability leave

401(k) retirement plan

Paid vacation

Paid holidays

Job summary

U.S. Bank is seeking a senior Reliability Engineer specializing in observability to translate customer journeys into measurable reliability objectives. You will lead the creation and governance of SLIs, SLOs, dashboards, and alerts across complex enterprise apps, ensuring instrumented systems and data-driven reliability decisions.

Collaboration with product, SRE, and operations is essential. The role emphasizes incident analysis, reducing alert fatigue, and mentoring teams on tracing, metrics,

Qualifications

• Six to eight years of relevant work experience in IT and reliability.
• Strong knowledge of SLIs, SLOs, and error budgets.
• Experience building dashboards and governance for observability assets.

Responsibilities

• Lead the definition, documentation, implementation, and continuous improvement of Observability across Critical Customer Journeys, ensuring alignment between Observability Strategy, business outcomes, and reliability objectives.
• Design, implement, and govern SLIs, SLOs, Error Budgets, and reliability metrics for enterprise applications and services.
• Establish and maintain Observability Governance Frameworks for Dashboards, Alerts, Synthetic Monitoring, Telemetry Standards, and lifecycle management of observability assets.
• Translate business and technical requirements into scalable Observability Architectures, including Instrumentation Standards, Monitoring Strategies, Tagging Frameworks, and Alerting Models.
• Partner with Product Owners, Application Engineering, Site Reliability Engineering (SRE), and Operations Teams to ensure applications are production-ready and fully instrumented for reliability measurement.
• Develop and maintain executive and operational Service Health Dashboards that provide insights into Availability, Latency, Customer Impact, Dependency Performance, and SLO Compliance.
• Analyze Telemetry Data, Incident Trends, Problem Records, and Alert Performance to identify observability gaps, reduce alert fatigue, and improve detection accuracy.
• Provide technical leadership and mentorship on Distributed Tracing, Logging, Metrics Collection, Synthetic Testing, Application Performance Monitoring, Real User Monitoring, Monitoring Design Patterns, and Alert Governance Best Practices.
• Lead the definition, documentation, and ongoing refinement of critical user journeys in partnership with product owners, engineering teams, SRE, operations, and business stakeholders to ensure observability practices are aligned to customer experience, business outcomes, and operational risk.
• Define, document, and govern appropriate service-level indicators and service-level objectives for applications and key capabilities, including availability, latency, error rate, throughput, dependency health, and other measurements that reflect meaningful customer and business impact.
• Establish and maintain a best-practice process for identifying, approving, implementing, reviewing, and retiring user journeys, SLIs, SLOs, dashboards, monitors, synthetic tests, alerts, and related observability artifacts.
• Translate product and engineering requirements into actionable observability designs that specify telemetry needs, measurement methods, tagging standards, dashboard requirements, alerting thresholds, ownership, evidence expectations, and operational runbook linkages.
• Partner with product and engineering teams during design, build, release, and production-readiness activities to ensure applications are instrumented to validate critical customer journeys, measure reliability outcomes, and support effective incident detection and triage.
• Develop, maintain, and continuously improve dashboards and reporting that communicate service health, SLO performance, error-budget posture, customer impact, dependency performance, alert effectiveness, and trends to technical teams and leadership stakeholders.
• Analyze telemetry, incidents, problem records, alert history, customer-impacting events, and SLO performance trends to identify observability gaps, reduce alert noise, improve detection accuracy, and recommend reliability improvements.
• Provide senior-level guidance, coaching, and standards interpretation to engineering, SRE, and operations teams on observability design patterns, SLI/SLO selection, customer journey monitoring, synthetic monitoring, logging, tracing, metrics, and alert governance.
• Maintain an authoritative inventory of observability assets, including user journeys, SLIs, SLOs, dashboards, monitors, alerts, synthetic tests, ownership assignments, review cadence, and evidence of ongoing compliance with approved observability standards.

Skills

Observability Engineering

SRE/Reliability

SLIs/SLOs

Distributed Tracing

Monitoring & Alerts

Telemetry/Logging

Stakeholder Communication

Mentorship

Incident Analysis

Education

Bachelor's degree

Tools

Datadog

Dynatrace

Splunk

Grafana

Prometheus

New Relic

Elastic

OpenTelemetry

Job description

U.S. Bank is seeking a senior Reliability Engineer specializing in observability to translate customer journeys into measurable reliability objectives. You will lead the creation and governance of SLIs, SLOs, dashboards, and alerts across complex enterprise apps, ensuring instrumented systems and data-driven reliability decisions.

Collaboration with product, SRE, and operations is essential. The role emphasizes incident analysis, reducing alert fatigue, and mentoring teams on tracing, metrics,