Principal Site Reliability Engineer - Paze

Early Warning Services LLC; San Francisco (CA); Hybrid; USD 194,000 - 284,000; Full time

Benefits offered by this job

Healthcare coverage

401(k) with company match

Paid time off

Job summary

Early Warning Services LLC is seeking a Principal Site Reliability Engineer to lead reliability across production systems. The role partners with Software Engineering and other tech teams to ensure performance, observability, recoverability, and operational readiness across the lifecycle.

The position requires deep technical leadership, strong debugging skills, and a track record applying evidence-driven engineering practices to enterprise-scale services. Hybrid work locations include major U.S.

Qualifications

• Typically 15+ years of relevant professional experience in Software Engineering, SRE, or related fields.
• Experience with software development or scripting using one or more modern programming languages.
• Experience with software engineering principles, distributed systems, production troubleshooting, automation, and observability.
• Experience with public cloud technologies (preferably AWS) and Linux/Unix.
• Strong analytical, problem-solving, and collaboration skills.

Responsibilities

• Provide enterprise-level technical leadership for critical production incidents and drive incident response improvements.
• Define and implement reliability metrics (SLIs/SLOs) and error budgets.
• Lead or participate in incident response, troubleshooting, and service restoration activities.
• Collaborate with software engineering to improve deploys, observability, and resiliency across lifecycles.
• Advance CI/CD, IaC, containerization, and automation practices across the organization.

Skills

Software engineering

Distributed systems

Troubleshooting

Communication

Collaboration

Education

Bachelor's degree in CS/Software Eng

Tools

AWS

CI/CD

Infrastructure as Code

Job description

At Early Warning, we’ve powered and protected the U.S. financial system for over thirty years with cutting-edge solutions like Zelle, Paze, and so much more. As a trusted name in payments, we partner with thousands of institutions to increase access to financial services and protect transactions for hundreds of millions of consumers and small businesses. Positions located in Scottsdale, San Francisco, Chicago, or New York follow a hybrid work model to allow for a more collaborative working environment. Candidates responding to this posting must independently possess the eligibility to work in the United States for any employer, at the date of hire. This position is ineligible for employment Visa sponsorship.

Role Summary

The Principal Site Reliability Engineer applies software engineering and systems engineering practices to improve the reliability, resilience, scalability, and operational health of production services. The role partners with Software Engineering and other technology teams to ensure reliability, observability, recoverability, performance, and operational readiness are engineered into systems throughout their lifecycle. The role operates at enterprise scope, establishing technical direction and applying evidence-driven engineering, technical rigor, sound judgment, automation, and broad systems expertise across organizational boundaries.

Core Responsibilities

• Use software engineering, automation, and DevOps principles and practices to continually improve how services are built, tested, deployed, observed, operated, and recovered.
• Use data, evidence, experimentation, and rigorous engineering analysis appropriate to the level to identify reliability risks, test assumptions, and guide technical decisions.
• Define, implement, or improve SLIs, SLOs, error budgets, and other service-health measures appropriate to the scope of responsibility.
• Improve observability through metrics, logging, tracing, monitoring, alerting, dashboards, and service-health instrumentation.
• Drive continuous improvement across CI/CD, observability, deployment practices, Infrastructure as Code, automation, testing, incident response, capacity management, resilience, and operational readiness.
• Identify recurring or systemic production issues and translate operational experience into improvements in code, architecture, automation, tooling, and engineering practices.
• Partner with Software Engineering teams to incorporate reliability, resiliency, scalability, performance, observability, recoverability, and operational readiness throughout the development lifecycle.
• Participate in or lead incident response, troubleshooting, service restoration, and blameless post-incident learning appropriate to the level.
• Provides enterprise-level technical leadership for critical production incidents and establishes or influences engineering practices that improve incident response, escalation, service restoration and sustainable on-call operations across the organization.
• Reduce operational toil and unnecessary manual intervention through software, automation, reusable patterns, and better engineering practices.

Leveling Intent

Principal represents domain-level technical leadership and organizational impact. Deep individual expertise is expected, but Principal-level impact comes from identifying systemic risk, establishing technical direction, influencing engineering practices and architecture, and multiplying the capability of the broader engineering organization.

Level Expectations

Acts as an enterprise force multiplier, raising the effectiveness and technical capability of engineers and teams across the organization while building sustainable organizational capability rather than individual dependency. Demonstrates software engineering, systems thinking, troubleshooting, and production reliability capabilities appropriate to the level. Applies evidence-driven reasoning and technical rigor to distinguish observed facts from assumptions and make defensible engineering recommendations. Shares knowledge and contributes to sustainable engineering capability rather than creating dependency on individual expertise. Operates with significant autonomy across the organization’s most consequential reliability challenges. Establishes enterprise technical direction, develops senior technical leaders, and demonstrates impact well beyond systems personally touched.

Minimum Qualifications

• Typically 15+ years of relevant professional experience in Software Engineering, Site Reliability Engineering, Systems Engineering, Cloud/Platform Engineering, DevOps, Infrastructure Engineering, Architecture where applicable, or a comparable technical discipline.
• Experience with software development or scripting using one or more modern programming languages.
• Experience with software engineering principles, distributed systems, production troubleshooting, automation, and observability appropriate to the level.
• Experience with public cloud technologies and architectures, preferably AWS, along with infrastructure, networking, Linux/Unix, and modern application architectures appropriate to the level.
• Demonstrated analytical, problem-solving, communication, and collaboration skills appropriate to the scope of the role.

Preferred Qualifications

• Hands-on experience with AWS is preferred, or comparable experience with another major cloud platform such as Microsoft Azure, Google Cloud Platform (GCP), or Oracle Cloud Infrastructure (OCI).
• Experience developing, deploying, operating, or improving highly available production software or distributed systems.
• Experience with CI/CD, Infrastructure as Code, containers or orchestration, observability, monitoring, alerting, and software-delivery automation.
• Experience with SLIs, SLOs, error budgets, incident management, performance analysis, capacity management, resilience testing, disaster recovery, or operational readiness appropriate to the level.
• Experience creating reusable automation, tooling, platforms, patterns, or practices that improve engineering effectiveness.
• Bachelor's degree in Computer Science, Software Engineering, Computer Engineering, Information Systems, or a related technical field, or equivalent practical experience.

The base pay scale for this position in: Phoenix, AZ/ Chicago, IL / Washington, DC in USD per year is: $194,000 - $237,000. New York, NY/ San Francisco, CA in USD per year is: $232,000 - $284,000. Additionally, candidates are eligible for a discretionary incentive plan and benefits. This pay scale is subject to change and is not necessarily reflective of actual compensation that may be earned, nor a promise of any specific pay for any specific candidate, which is always dependent on legitimate factors considered at the time of job offer. Early Warning Services takes into consideration a variety of factors when determining a competitive salary offer, including, but not limited to, the job scope, market rates and geographic location of a position, candidate’s education, experience, training, and specialized skills or certification(s) in relation to the job requirements and compared with internal equity (peers). The business actively supports and reviews wage equity to ensure that pay decisions are not based on gender, race, national origin, or any other protected classes.

Some of the Ways We Prioritize Your Health and Happiness

• Healthcare Coverage - Competitive medical (PPO/HDHP), dental, and vision plans as well as company contributions to your Health Savings Account (HSA) or pre-tax savings through flexible spending accounts (FSA) for commuting, health and dependent care expenses.
• 401(k) Retirement Plan - Featuring a 100% Company Safe Harbor Match on your first 6% deferral immediately upon eligibility.
• Paid Time Off - Flexible Time Off for Exempt (salaried) employees, as well as generous PTO for Non-Exempt (hourly) employees, plus 11 paid company holidays and a paid volunteer day.
• 12 weeks of Paid Parental Leave
• Maven Family Planning - provides support through your Parenting journey including egg freezing, fertility, adoption, surrogacy, pregnancy, postpartum, early pediatrics, and returning to work.
• And SO much more! We continue to enhance our program, so be sure to check our Benefits page here for the latest. Our team can share more during the interview process!

Physical Requirements

• Early Warning works together in a highly collaborative office environment.
• Working conditions consist of a normal office environment.
• Work is primarily sedentary and requires extensive use of a computer and involves sitting for periods of approximately four hours.
• Work may require occasional standing, walking, kneeling, and reaching.
• Must be able to lift 10 pounds occasionally and/or negligible amount of force frequently.
• Requires visual acuity and dexterity to view, prepare, and manipulate documents and office equipment including personal computers.
• Requires the ability to communicate with internal and/or external customers.
• Employee must be able to perform essential functions and physical requirements of position with or without reasonable accommodation.
• Candidates responding to this posting must independently possess the eligibility to work in the United States at the date of hire.

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Early Warning Services, LLC (“Early Warning”) considers for employment, hires, retains and promotes qualified candidates on the basis of ability, potential, and valid qualifications without regard to race, religious creed, religion, color, sex, sexual orientation, genetic information, gender, gender identity, gender expression, age, national origin, ancestry, citizenship, protected veteran or disability status or any factor prohibited by law, and as such affirms in policy and practice to support and promote equal employment opportunity and affirmative action, in accordance with all applicable federal, state, and municipal laws.

The company also prohibits discrimination on other bases such as medical condition, marital status or any other factor that is irrelevant to the performance of our employees.

View supplemental EEO is the law poster Early Warning has developed and maintains a written AAP and upholds pay transparency nondiscrimination.

E-Verify Early Warning Services LLC is a proud participant in E-Verify, a federal program to help ensure a legal and authorized workforce. As part of our hiring process, we electronically verify the employment eligibility of all new hires through E-Verify. For more information on your rights and responsibilities under E-Verify please visit Home | E-Verify.

Early Warning Services delivers innovative payment and risk solutions to financial institutions nationwide. For over 25 years, Early Warning has been a leader in technology that helps protect and advance the financial system. We serve a diverse network of approximately 2,500 financial institutions, government entities and payment companies. Our product solutions enable real-time funds availability for a variety of payment types through our payments network.