Engineer, Site Reliability

Vanguard; Malvern (Chester County); On-site; USD 140,000 - 200,000; Full time

Job summary

Vanguard is seeking a Staff Reliability Engineer to lead PWTech reliability initiatives, architecting enterprise-scale resiliency solutions and automating incident responses. You will drive observability, distributed tracing, and diagnostics across cloud-native stacks, while influencing teams and setting standards for reliability from inception.

The role requires deep experience in distributed systems, leadership, and hands-on development in AWS, OpenTelemetry, and Python, with a strong emphasis

Qualifications

• Eight+ years related experience, with at least two years of development experience.
• Undergraduate degree or equivalent combination of training and experience. Graduate degree preferred.

Responsibilities

• Lead the technical strategy, architecture, and evolution of PWTech reliability engineering platforms and capabilities, ensuring they scale across hundreds of applications and client-facing systems.
• Design and build production-grade software and platforms that improve reliability outcomes, including automated incident detection, diagnostics, remediation, resiliency engineering, and operational intelligence.
• Drive enterprise observability and diagnostics capabilities, enabling telemetry, distributed tracing, metrics, and operational insights across cloud-native technologies and application stacks.
• Define and codify resilient application and platform patterns, such as graceful degradation, circuit breakers, load shedding, fault isolation, failover, and automated recovery, driving adoption through reusable software and standards.
• Influence engineering teams and technology leaders to embed reliability from inception and establish technical standards across the organization.
• Lead the resolution of complex reliability and production challenges, performing root cause analysis and engineering durable solutions.
• Participate in special projects as assigned.

Skills

Distributed systems

Leadership

Root cause analysis

System design

Communication

Education

Bachelor's degree or equivalent

Tools

AWS

OpenTelemetry

Python

Job description

Join the Personal Wealth Technology (PWTech) Reliability Engineering team and lead cutting-edge Reliability Engineering initiatives that impact hundreds of applications and millions of investors. You’ll architect and build enterprise-scale resiliency solutions, driving our ambitious roadmap. This is an opportunity to combine deep technical expertise with strategic influence —automating incident responses, implementing distributed tracing at scale, and pioneering AI-enhanced diagnostics and analysis. Work alongside a collaborative, technically-focused team where your innovation in resilience engineering will shape Vanguard’s next generation of client experiences.

At Vanguard, we pride ourselves on delivering an exceptional client experience to all investors; at the core of this experience are systems that reside in a technically complex and constantly evolving resiliency landscape. Passionate, technically skilled engineers are at the center of our resiliency operations, and we are looking to grow our team.

We are seeking an experienced engineer with broad, end-to-end software development experience, including operating applications in a microservices environment in production at scale. This role goes beyond feature implementation - it requires someone who can design, build, and support resilient systems from the ground up.

As a Staff Reliability Engineer at Vanguard, you will play a critical role in solving impactful operational problems. You are curious and take a proactive approach to identifying problems and making improvements. You balance innovative thinking with pragmatism and understand the long-term impacts of technical decisions. You communicate complex ideas clearly and collaborate effectively to deliver scalable solutions.

Core Responsibilities

• Lead the technical strategy, architecture, and evolution of PWTech reliability engineering platforms and capabilities, ensuring they scale across hundreds of applications and critical client-facing systems.
• Design and build production-grade software and platforms that improve reliability outcomes, including automated incident detection, diagnostics, remediation, resiliency engineering, and operational intelligence.
• Drive enterprise observability and diagnostics capabilities, enabling consistent telemetry, distributed tracing, metrics, and operational insights across cloud-native technologies and application stacks.
• Define and codify resilient application and platform patterns, such as graceful degradation, circuit breakers, load shedding, fault isolation, failover, and automated recovery, driving adoption through reusable software, frameworks, and engineering standards.
• Influence engineering teams and technology leaders across the organization, establishing technical standards and ensuring reliability is designed into systems from inception.
• Lead the resolution of complex reliability and production challenges, identifying systemic risks, driving root cause analysis, and engineering durable solutions that improve long-term resilience.
• Participates in special projects and performs other duties as assigned.

Qualifications

• Minimum of eight years related experience, with at least two years of development experience.
• Undergraduate degree or equivalent combination of training and experience. Graduate degree preferred.

Preferred Skills

• Experience designing, building, and operating production-facing platforms or engineering capabilities that achieve broad adoption and deliver measurable reliability, operational, or business outcomes.
• Deep expertise in distributed systems architecture, including scalability, availability, resiliency, fault tolerance, performance optimization, and production operations at scale.
• Strong technical leadership and influence skills, with a demonstrated ability to drive architecture decisions, establish technical standards, and align multiple teams on engineering direction.
• Demonstrated ability to diagnose and resolve complex production issues, perform root cause analysis, and engineer durable solutions that prevent recurrence.
• Hands‑on experience with AWS and modern cloud architecture patterns.
• Experience with observability and telemetry platforms, including metrics, logging, distributed tracing, and production diagnostics. Experience with OpenTelemetry is strongly preferred.
• Proficiency with Python or similar scripting languages to develop automation, tooling, and operational workflows.
• Strong software engineering fundamentals, systems thinking skills, experience solving complex reliability challenges, and the ability to influence across teams and drive engineering best practices