Senior SRE: Observability & Cloud Platform Engineer

Cisco; San Francisco (CA); On-site; USD 168,000 - 245,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401(k) with Cisco matching

Paid parental leave

Disability coverage

Life insurance

Flexible vacation time off

Job summary

Cisco is seeking an experienced Site Reliability Engineer to design, deploy, and operate enterprise observability platforms across cloud-scale infrastructure. You’ll lead efforts to improve visibility, reduce incident resolution times, and strengthen reliability in production services.

You will build dashboards and alerts using Splunk, Grafana, and Tempo, while supporting distributed tracing and end-to-end telemetry pipelines.

Qualifications

• 7+ years of experience in Site Reliability Engineering, Platform Engineering, DevOps, or a related field.
• Experience administering Splunk Enterprise or Splunk Cloud in a production environment.
• Solid understanding of Splunk SPL and experience developing production dashboards, alerts, and analytics.
• Experience designing, deploying, and operating Elasticsearch or ELK-based platforms.
• Experience with Prometheus, Grafana, Grafana Tempo, OpenTelemetry, distributed tracing, and Kafka.
• Demonstrated experience implementing and operating modern observability strategies across logs, metrics, and traces.
• Experience with Terraform and Infrastructure as Code practices.
• Strong understanding of Linux, networking, distributed systems, high availability, and production operations.
• Programming or scripting experience in Python, Go, Ruby, Bash, or a similar language.
• Experience troubleshooting production incidents and working within an on-call or operational support model.
• Strong communication, collaboration, and problem‑solving skills.

Responsibilities

• Design, deploy, operate, and continuously improve enterprise observability platforms.
• Build and maintain Splunk Enterprise and Splunk Cloud infrastructure, including indexers, Search Head Clusters, heavy forwarders, deployment servers, and related integrations.
• Deploy and operate large-scale Elasticsearch clusters supporting log analytics, search, and operational troubleshooting.
• Design, deploy, and support distributed tracing platforms using Grafana Tempo and OpenTelemetry.
• Build and maintain end-to-end telemetry pipelines for logs, metrics, and traces.
• Define instrumentation standards, data quality practices, retention policies, and observability patterns.
• Scale and optimize Prometheus, Grafana, Kafka, Tempo, OpenTelemetry, and related monitoring systems.
• Develop dashboards, alerts, analytics, and trace visualizations using Splunk SPL, Grafana, Kibana, and Tempo.
• Establish monitoring and alerting standards that improve service-level visibility and reduce mean time to detect and resolve incidents.
• Automate infrastructure provisioning, configuration, upgrades, and operational processes using Terraform and configuration management tools.
• Participate in capacity planning, performance tuning, upgrades, patching, disaster recovery, and production readiness reviews.
• Troubleshoot complex distributed-systems issues and lead or support incident response and root-cause analysis.
• Partner with platform, application, security, database, and network engineering teams to improve reliability and operational consistency.
• Contribute to shared SRE standards, documentation, runbooks, and engineering best practices.
• Participate in the shared SRE pager and on-call rotation for production services.

Skills

Site Reliability Engineering

Platform Engineering

DevOps

Splunk

Splunk SPL

Elasticsearch

OpenTelemetry

Kafka

Prometheus

Grafana

Terraform

Python

Go

Bash

Ruby

Linux

Kubernetes

CI/CD

AWS

Azure

GCP

Tools

Splunk Enterprise

Splunk Cloud

Elasticsearch

Kibana

Grafana

Tempo

OpenTelemetry

Kafka

Kubernetes

Terraform

Ansible

AWS

Azure

GCP

Python

Go

CI/CD

Linux

Job description

Cisco is seeking an experienced Site Reliability Engineer to design, deploy, and operate enterprise observability platforms across cloud-scale infrastructure. You’ll lead efforts to improve visibility, reduce incident resolution times, and strengthen reliability in production services.

You will build dashboards and alerts using Splunk, Grafana, and Tempo, while supporting distributed tracing and end-to-end telemetry pipelines.