We are seeking a highly skilled and visionary Principal/Lead Site Reliability Engineer (SRE) to spearhead our critical migration from legacy systems to the Epic platform. In this technical leadership role, you will act as the driving force behind our system resilience, designing end-to-end observability, and architecting fault-tolerant solutions across a highly complex, multi-layered ecosystem.
As a Principal Site Reliability Engineer, you will be responsible for building a SRE practice, monitoring and performance engineering best practices which will be aligned to our agile teams to help drive availability, resiliency and stability of Quest products, platform and services.
You will also be a partner with architecture, engineers, security, and operations to design and build reusable patterns to deploy reliable and resilient solutions.
Duties and Responsibilities:
- Analyze and resolve complex technical challenges to ensure a seamless and secure migration process.
- Translate functional business knowledge into robust, actionable observability strategies.
- Design end-to-end system resilience plans across our entire ecosystem.
- Architect fault tolerance and implement comprehensive Disaster Recovery (DR) solutions.
- Navigate system limitations and security constraints to improve observability securely, ensuring strict adherence to data masking and access restriction policies.
- Engineer custom log ingestion pipelines, unblocking log flow by routing server logs to diverse tools (such as Dynatrace) rather than relying solely on direct sourcing.
- Manage and optimize Azure Monitoring solutions across our cloud infrastructure.
- Build and provision scalable infrastructure solutions utilizing Infrastructure as Code (IaC) tools such as Terraform and Ansible.
- Develop robust automation scripts using Python, JavaScript, or Shell to streamline operations.
- Administer observability platforms within a highly complex environment, bringing our monitoring practices to the next stage of maturity.
- Build comprehensive observability frameworks tailored specifically for middleware environments within multi-layered architectures.
- Implement DevSecOps practices by integrating CI/CD pipelines (e.g., GitLab) and automated security scanning tools.
- Mentor, guide, and encourage the engineering team to adopt new enhancements and automation within the observability space.
Required Work Experience:
- 10+ years of experience as SRE lead/ Principle SRE and Hands on multiple languages e.g., Java and web technologies (Python, Go, Perl, Ruby or shell scripting)
- 4+ experience as a Developer/ Automation Engineer
- Expert knowledge of reliability and production management domains
- Experience in public cloud environments (AWS/Azure/Google Cloud).
- Experience with cloud-based technologies and tools especially in deployment, monitoring and operations
- Strong experience and technical skills in developing/managing APIs and Microservices
- Expert practitioner in multiple technology domains, may be a cross-domain expert able to solve complex and mission critical problems within a business or across the firm
- Analyze and resolve complex technical challenges to ensure a seamless and secure migration process.
- Translate functional business knowledge into robust, actionable observability strategies.
- Design end-to-end system resilience plans across our entire ecosystem.
- Architect fault tolerance and implement comprehensive Disaster Recovery (DR) solutions.
- Navigate system limitations and security constraints to improve observability securely, ensuring strict adherence to data masking and access restriction policies.
- Engineer custom log ingestion pipelines, unblocking log flow by routing server logs to diverse tools (such as Dynatrace) rather than relying solely on direct sourcing.
- Manage and optimize Azure Monitoring solutions across our cloud infrastructure.
- Build and provision scalable infrastructure solutions utilizing Infrastructure as Code (IaC) tools such as Terraform and Ansible.
- Develop robust automation scripts using Python, JavaScript, or Shell to streamline operations.
- Administer observability platforms within a highly complex environment, bringing our monitoring practices to the next stage of maturity.
- Build comprehensive observability frameworks tailored specifically for middleware environments within multi-layered architectures.
- Implement DevSecOps practices by integrating CI/CD pipelines (e.g., GitLab) and automated security scanning tools.
- Mentor, guide, and encourage the engineering team to adopt new enhancements and automation within the observability space.
Preferred Work Experience:
- Experience with containerization (Docker, Kubernetes)
- Experience with Terraform and Ansible
- Experience with SEIM
- Experience with other APM tools (Administartion)
- Healthcare industry experience
- Azure cloud, Dynatrace Observability
Physical and Mental Requirements:
- Ability to sit for long periods of time
Knowledge:
- Cloud
- APM
- Technical Architecture
- Disaster Recovery/ Chaos Engineering
Skills:
- Self-driven
- Problem solving
- Adaptable
- Negotiation
- Prioritization