Staff Production Engineer (SRE) (Federal)

Zscaler; Virginia (MN); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Zscaler is seeking a Staff Site Reliability Engineer (Production Engineer) for a hybrid role based in the San Jose area. You will own the systems-level reliability of high-throughput bare-metal and cloud infrastructure, debugging issues at OS and network level, and driving platform resilience.

Responsibilities include incident response, automation of lifecycle management, telemetry orchestration with Prometheus/OpenTelemetry, and CI/CD validation to ensure seamless production deployments.

Qualifications

• 5+ years of experience in Site Reliability Engineering, Production Engineering, or Systems Engineering operating high-scale, low-latency production platforms.
• Proven ability to write and debug executable code live (Python, Go, or Bash) covering core logic/data structures, along with hands-on experience writing Ansible playbooks/tasks for infrastructure automation.
• Deep knowledge of Linux OS internals and kernel troubleshooting and networking concepts; comfortable debugging at OS/network levels.
• US Citizenship is required due to sensitive customer engagements.

Responsibilities

• Maintain high availability across large-scale bare-metal or cloud-native infrastructure and Kubernetes clusters.
• Lead incident response with cross-stack troubleshooting using low-level OS and network tools.
• Automate infrastructure lifecycle management, service provisioning, configurations, and release deployments.
• Operate telemetry pipelines (metrics, logs, traces) using Prometheus and OpenTelemetry; define/review SLOs and error budgets.
• Perform capacity planning, OS/kernel upgrades, and CI/CD validation prior to production rollouts.

Skills

SRE

Python

Go

Bash

Linux

Kubernetes

Ansible

Telemetry

Tools

Prometheus

OpenTelemetry

Ansible

Job description

About Zscaler

Zscaler accelerates digital transformation to ensure our customers can be more agile, efficient, resilient, and secure. As an AI-forward enterprise, we are constantly pushing the envelope, leveraging the world's largest security data lake to power our cloud-native Zero Trust Exchange platform. This innovation protects our customers from cyberattacks and data loss by securely connecting users, devices, and applications in any location.

Here, impact in your role matters more than title and trust is built on results. We say, impact over activity. We seek innovators who actively use AI to amplify their impact and who thrive in an environment where we leverage intelligent systems to stay ahead of evolving threats. We believe in transparency and value constructive, honest debate –we're focused on getting to the best ideas, faster. We build high-performing teams that can make an impact quickly and with high quality. To do this, we are building a culture of execution centered on customer obsession, collaboration, ownership, and accountability.

We value high-impact, high-accountability with a sense of urgency where you're enabled to do your best work and embrace your potential. If you're driven by purpose, thrive on solving complex challenges, and want to be part of the team that's helping to secure the AI age, we invite you to bring your talents to Zscaler and help shape the future of cybersecurity.

Role

We are looking for a Staff Site Reliability Engineer (Production Engineer) to join our team. This is a hybrid role (onsite three days a week in San Jose, CA or another Zscaler office; remote can be considered for exceptional candidates) reporting to the Senior Manager, Site Reliability Engineering in the Zero Trust Exchange department. As a key member of the Zero Trust Exchange team, you will own the systems-level reliability and performance of Zscaler's high-throughput bare-metal and cloud infrastructure processing tens of billions of daily transactions. You will debug live production issues at the OS and network level, write production-grade automation, and drive platform resilience.

What you'll do (Role Expectations)

• Partner with Engineering and Networking teams to maintain high availability across large-scale bare-metal Linux/BSD fleets, Kubernetes clusters, and custom routing stacks.
• Lead full-cycle incident response by conducting cross-stack troubleshooting using low-level OS and network tools (strace, lsof, tcpdump, iostat, vmstat, gdb).
• Automate infrastructure lifecycle management, service provisioning, configuration workflows, and release deployments using Ansible, Python, and Bash.
• Operate end-to-end telemetry (metrics, logs, traces) using Prometheus and OpenTelemetry ecosystems; define and enforce SLOs/error budgets to reduce alert noise.
• Perform architectural reviews, OS/kernel upgrades, capacity and performance tuning, and strict CI/CD validation prior to production rollouts.

Who You Are (Success Profile)

• You thrive in ambiguity. You're comfortable building the path as you walk it. You thrive in a dynamic environment, seeing ambiguity not as a hindrance, but as the raw material to build something meaningful.
• You act like an owner. Your passion for the mission fuels your bias for action. You operate with integrity because you genuinely care about the outcome. True ownership involves leveraging dynamic range: the ability to navigate seamlessly between high-level strategy and hands-on execution.
• You are a problem-solver. You love running towards the challenges because you are laser-focused on finding the solution, knowing that solving the hard problems delivers the biggest impact.
• You are a high-trust collaborator. You are ambitious for the team, not just yourself. You embrace our challenge culture by giving and receiving ongoing feedback-knowing that candor delivered with clarity and respect is the truest form of teamwork and the fastest way to earn trust.
• You are a learner. You have a true growth mindset and are obsessed with your own development, actively seeking feedback to become a better partner and a stronger teammate. You love what you do and you do it with purpose.

What We're Looking for (Minimum Qualifications)

• US Citizenship is required (due to the nature of assigned customers)
• 5+ years of experience in Site Reliability Engineering, Production Engineering, or Systems Engineering operating high-scale, low-latency production platforms.
• Proven ability to write and debug executable code live (Python, Go, or Bash) covering core logic/data structures, along with hands-on experience writing Ansible playbooks/tasks for infrastructure automation.
• Deep knowledge of Linux OS internals and kernel troubleshooting (e.g., inodes, open file descriptors, process states, and analyzing df vs du storage discrepancies). Comprehensive understanding of networking protocols and packet-level analysis, includ