SRE Devops Engineer

Persistent; Hyderabad, Bengaluru; On-site; INR 3,000,000 - 6,000,000; Full time

Benefits offered by this job

Competitive salary

Education sponsorship and growth

Cutting-edge technologies exposure

Flexible work hours and events

Annual health check-ups

Insurance coverage: group life and med

Job summary

Persistent is seeking an SRE DevOps Engineer to build and maintain enterprise observability platforms and improve service reliability across cloud-native environments. You will design dashboards, manage alerting, and drive observability-as-code with OpenShift, Kubernetes, Grafana, and Prometheus.

The role emphasizes incident response, data accuracy, and collaboration with cross-functional teams to enhance platform reliability and operational excellence in a hybrid work setup.

Qualifications

• 8 to 10 years in SRE/DevOps or production operations.
• Strong experience with enterprise observability platforms.
• Deep knowledge of SRE principles and reliability practices.
• Experience with OpenShift and Kubernetes architectures.
• Proficient in Python and Shell scripting.

Responsibilities

• Design, develop, and maintain Grafana dashboards using dashboards-as-code.
• Create, tune, and manage Prometheus/Mimir alert rules.
• Deploy and manage observability components via APIs and automation scripts.
• Validate metrics and data accuracy before production deployment.
• Lead production incident triage and root cause analysis.
• Develop automation for dashboard generation and alerts.
• Maintain runbooks and operational docs.

Skills

SRE/DevOps

Observability

Grafana dashboards

Prometheus

OpenShift

Kubernetes

Python scripting

Shell scripting

Incident response

Tools

Grafana

Prometheus

Mimir

Thanos

OpenShift

Kubernetes

Splunk

Python

Shell

Job description

AboutPosition:

We are seeking a highly skilled SRE Devops Engineer to build and maintain enterprise observability platforms, improve service reliability, and enhance operational excellence across cloud-native and containerized environments. The ideal candidate will possess strong expertise in OpenShift, Kubernetes, Grafana, Prometheus, DevOps practices, observability engineering, and production support. You will play a key role in defining reliability metrics, reducing alert fatigue, improving incident response capabilities, and driving observability-as-code initiatives across the organization.

• Role: SRE Devops Engineer
• Location: Hyderabad, Bengaluru
• Experience: Between 8 to 10 Years
• Job Type: Full Time Employment

WhatYou'llDo:

• Design, develop, and maintain Grafana dashboards using dashboards-as-code and JSON-based implementations.
• Build executive, operational, tenant-capacity, infrastructure, and service-level monitoring dashboards.
• Create, tune, and manage Prometheus/Mimir alert rules, notification policies, contact points, and mute schedules.
• Deploy and manage observability components through APIs, automation scripts, and provisioning frameworks.
• Validate PromQL queries, metrics availability, and data accuracy before deployment into production environments.
• Drive production incident triage and root cause analysis using Grafana, Splunk, OpenShift, and Kubernetes monitoring tools.
• Correlate metrics, logs, traces, and platform health data to identify issues rapidly.
• Reduce alert fatigue through threshold tuning, notification optimization, routing strategies, and incident-driven refinement.
• Define, implement, and improve Service Level Indicators (SLIs), Service Level Objectives (SLOs), and Error Budgets.
• Develop Python and Shell-based automation solutions for dashboard generation, alert deployment, and operational tasks.
• Maintain technical documentation including runbooks, troubleshooting guides, architecture decisions, and operational procedures.
• Partner with development, infrastructure, platform, security, and operations teams to improve overall platform reliability.
• Contribute to observability architecture reviews and reliability engineering best practices.
• Drive adoption of observability-as-code and automation-first approaches across engineering teams.

ExpertiseYou'llBring:

• 8 to 10 years of experience in Site Reliability Engineering, DevOps, Platform Engineering, or Production Operations.
• Strong experience designing and supporting enterprise observability platforms.
• Deep understanding of SRE principles, service reliability, and operational excellence practices.
• Strong expertise in Grafana dashboard development and observability-as-code implementations.
• Hands-on experience with Prometheus, Mimir, Thanos, and cloud-native monitoring ecosystems.
• Experience developing automation solutions using Python and Shell scripting.
• Knowledge of distributed systems, scalability patterns, and production support methodologies.
• Experience handling major incidents, production triage, and root cause analysis.
• Strong expertise in PromQL including joins, recording rules, rate/irate functions, histogram_quantile, label matching, and subqueries.
• Deep understanding of Prometheus, Mimir, and Thanos architecture.
• Experience with metric collection strategies, scrape configuration management, and cardinality optimization.
• Ability to validate metrics, labels, and query outputs before production deployment.
• Experience authoring dashboard JSON configurations directly.
• Expertise in Grafana panel schema design, transformations, templating, variables, and dashboard provisioning.
• Familiarity with Grafana classic dashboards and modern Grafana v2/scenes architecture.
• Experience provisioning dashboards, alerts, contact points, and notification policies using REST APIs and automation frameworks.
• Strong Git-based workflow experience with version-controlled observability assets.
• Extensive experience authoring actionable alert rules and notification strategies.
• Experience configuring notification policies, routing trees, contact points, and escalation workflows.
• Expertise in reducing noise through alert grouping, silencing strategies, and threshold optimization.
• Experience designing alerting systems based on real production incidents and postmortem outcomes.
• Strong understanding of SLI, SLO, Error Budget, and reliability governance frameworks.
• Strong hands-on experience with OpenShift Container Platform (OCP).
• Deep understanding of Kubernetes architecture, namespaces, pods, resource quotas, and cluster capacity planning.
• Experience with kube-state-metrics, node-exporter, and Kubernetes observability tooling.
• Understanding of multi-tenant observability models, chargeback monitoring, and governance dashboards.

Benefits:

• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Annual health check-ups
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven,People-Centric&InclusiveWorkEnvironment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment.

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."