Principal Data Platform Engineer – Real-Time Analytics

Palo Alto Networks, Inc.; Santa Clara (CA); On-site; USD 147,000 - 238,000; Full time

Job summary

Palo Alto Networks, Inc. is seeking a Principal Software Engineer, Data Platform on the Prisma AIRS Runtime Security team.

You will serve as a technical visionary and strategic leader shaping our enterprise data infrastructure, collaborating with executives, product management, ML engineers, data scientists, and cross‑functional software teams to design and deliver cutting‑edge streaming and batch processing solutions.

Qualifications

• Bachelor's degree in Computer Science or a related field with 10+ years of relevant industry experience, or a Master's degree with 8+ years of experience.
• Deep domain expertise in designing, building, and operating highly scalable, fault‑tolerant distributed systems, with exceptional programming skills in Golang and Python.
• Hands‑on experience building backend data platform infrastructure using distributed execution engines such as Apache Beam or Apache Spark.
• Hands‑on experience architecting, building, or deploying complex agentic workflows, tool‑use pipelines, or multi‑agent orchestrations within data infrastructure.
• Proven track record of profiling, benchmarking, and tuning high‑throughput distributed pipelines to improve latency and resource efficiency.
• Broad experience with cloud‑native data stacks (GCP/Dataflow preferred), messaging (Pub/Sub, Kafka), storage engines (BigQuery, Iceberg), and infrastructure tooling (Kubernetes, Terraform).
• Demonstrated ability to drive consensus across multi‑team boundaries, solve complex technical problems, and mentor senior engineering talent.

Responsibilities

• Partner with Product Management, SRE, ML Engineering, Security Researchers, and Quality Engineering to design and deliver high‑throughput data platform capabilities and streaming services.
• Architect and build resilient, multi‑source batch and streaming engines to power analytical workflows, business insights, and internal feature improvements.
• Profile, benchmark, and tune distributed execution DAGs, state storage, and compute resources to maximize throughput and lower latency.
• Evangelize and implement modern software engineering practices, including spec‑driven development and comprehensive testing frameworks.
• Mentor senior and junior engineers, conduct architectural reviews, and foster technical excellence across the team.

Skills

Mentoring

Cross-team collaboration

Problem solving

Leadership

Communication

Education

Bachelor's degree in Computer Science or related field

Master's degree with 8+ years experience

Tools

Golang

Python

Apache Beam

Apache Spark

Kubernetes

Terraform

Dataflow (GCP)

Pub/Sub

Kafka

BigQuery

Iceberg

Job description

Palo Alto Networks, Inc. is seeking a Principal Software Engineer, Data Platform on the Prisma AIRS Runtime Security team.

You will serve as a technical visionary and strategic leader shaping our enterprise data infrastructure, collaborating with executives, product management, ML engineers, data scientists, and cross‑functional software teams to design and deliver cutting‑edge streaming and batch processing solutions.