Principal Data Pipeline Architect, Real-Time Security

Palo Alto Networks; California (MO); On-site; USD 156,000 - 253,000; Full time

Job summary

Palo Alto Networks is advancing an AI-driven security platform and is building a new AIRS data pipeline initiative. You will architect and scale data ingestion pipelines, ensuring real-time access to telemetry from security engines and supporting ML model training at scale.

The role focuses on Go/Python/Java, OpenTelemetry observability, and distributed backend systems, with collaboration across data engineering and ML teams in a fast-paced security environment.

Qualifications

• BS/MS in Computer Science or related field with 8+ years of software or data engineering experience.
• Experience building distributed, high-throughput streaming data pipelines.

Responsibilities

• Design, build, and scale high-throughput log and telemetry ingest pipelines with ultra-low latency.
• Implement and standardize observability using OpenTelemetry across microservices.
• Architect and manage a performant trace lake for deep network forensics and ML training.
• Develop and maintain benchmark infra to validate pipeline performance under peak loads.
• Collaborate with Data Plane and ML teams to ensure smooth data flow into analytical platforms.

Skills

Go

Python

Java

Education

BS/MS in Computer Science or related field

Tools

GCP

ClickHouse

Elasticsearch

Terraform

Kubernetes

Job description

Palo Alto Networks is advancing an AI-driven security platform and is building a new AIRS data pipeline initiative. You will architect and scale data ingestion pipelines, ensuring real-time access to telemetry from security engines and supporting ML model training at scale.

The role focuses on Go/Python/Java, OpenTelemetry observability, and distributed backend systems, with collaboration across data engineering and ML teams in a fast-paced security environment.