Staff Data Engineer (Data & AI solutions)

Jobs Paloaltonetworks; Palo Alto (CA); Hybrid; GBP 90,000 - 150,000; Full time

Job summary

Palo Alto Networks is seeking an experienced Senior Staff Data Engineer to drive our AI-forward data platform initiatives. You will own data lakehouse architectures, build scalable ETL/ELT pipelines, and set data engineering best practices across the organization.

As a technical project lead, collaborate with Data Science, Analytics, R&D, and IT to optimize onboarding workflows and accelerate cross-functional data initiatives.

Qualifications

• 3+ years in advanced data modeling (facts, dimensions, partitions).
• 3+ years ETL/ELT in Data Lakehouse architectures.
• 3+ years SQL/Python/PySpark/Java/Scala experience.
• Experience leading technical projects across teams.
• AWS or GCP cloud experience, with emphasis on AWS.

Responsibilities

• Own the design, implementation, and optimization of scalable Data Lakehouse and Data Warehouse architectures across diverse cloud ecosystems.
• Spearhead the development of robust ETL/ELT pipelines and data ingestion processes utilizing SQL, Python, PySpark, and DBT.
• Apply advanced data modeling techniques to structure complex data assets into optimized facts, dimensions, and partitions.
• Partner with cross-functional leadership across data science, analytics, R&D, and IT to identify requirements and improve workflows.
• Orchestrate complex data workflows combining tools like Airflow and DBT with advanced Gen-AI development workflows using CloudCode.
• Define, document, and popularize data engineering best practices, design patterns, and data quality standards across the unit.
• Evaluate and implement modern data technologies spanning vendor-specific and open-source landscapes including GCP, AWS, and Apache Iceberg.
• Provide technical direction and hands-on guidance to ensure high-performance execution, reliable pipeline delivery, and proactive problem-solving.

Skills

Advanced data modeling

ETL/ELT development

SQL

Python

PySpark

AWS

Gen-AI workflows

Leadership

Tools

Airflow

DBT

Iceberg

CloudCode

Cursor

Job description

Our Mission

At Palo Alto Networks®, we’re united by a shared mission—to protect our digital way of life. We thrive at the intersection of innovation and impact, solving real-world problems with cutting-edge technology and bold thinking. Here, everyone has a voice, and every idea counts. If you’re ready to do the most meaningful work of your career alongside people who are just as passionate as you are, you’re in the right place.

Who We Are

In order to be the cybersecurity partner of choice, we must trailblaze the path and shape the future of our industry. This is something our employees work at each day and is defined by our values: Disruption, Collaboration, Execution, Integrity, and Inclusion. We weave AI into the fabric of everything we do and use it to augment the impact every individual can have. If you are passionate about solving real-world problems and ideating beside the best and the brightest, we invite you to join us!

We believe collaboration thrives in person. That’s why most of our teams work from the office full time, with flexibility when it’s needed. This model supports real-time problem-solving, stronger relationships, and the kind of precision that drives great outcomes.

Job Summary

IDIRA by Palo Alto Networks seeks an experienced Senior Staff Data Engineer to join our AI and Data Solutions unit and champion an AI-first strategy. You will spearhead the development of Data Lakehouse processes, model core product domains, and define data engineering best practices across the organization. Serving as a technical project lead, you will provide strong direction and collaborate with Data Science, Analytics, R&D, and IT teams to elevate onboarding workflows. This role demands a proactive approach to optimizing system capabilities and driving cross-functional data initiatives.

Key Responsibilities

• Own the design, implementation, and optimization of scalable Data Lakehouse and Data Warehouse architectures across diverse cloud ecosystems.
• Spearhead the development of robust ETL/ELT pipelines and data ingestion processes utilizing SQL, Python, PySpark, and DBT.
• Apply advanced data modeling techniques to structure complex data assets into optimized facts, dimensions, and partitions.
• Partner with cross-functional leadership across data science, analytics, R&D, and IT to identify requirements and improve workflows.
• Orchestrate complex data workflows combining tools like Airflow and DBT with advanced Gen-AI development workflows using CloudCode.
• Define, document, and popularize data engineering best practices, design patterns, and data quality standards across the unit.
• Evaluate and implement modern data technologies spanning vendor-specific and open-source landscapes including GCP, AWS, and Apache Iceberg.
• Provide technical direction and hands-on guidance to ensure high-performance execution, reliable pipeline delivery, and proactive problem-solving.

Qualifications

Required Qualifications

• 3+ years of experience with advanced data modeling techniques and concepts (e.g., Facts, Dimensions, Partitions).
• 3+ years of experience designing and implementing end-to-end data ingestion and ETL processes within Data Lakehouse architectures.
• 3+ years of hands-on experience programming in SQL and Python, PySpark, Java, or Scala.
• Demonstrated experience leading technical projects, driving stakeholder alignment, and navigating cross-functional dynamics.
• Proven experience working within cloud environments (AWS or GCP), with a strong requirement for AWS.
• Deep understanding of the modern data landscape and open-source tools (e.g., Iceberg, DBT, Airflow).
• Proven capability in using and orchestrating Gen-AI workflows based on CloudCode and Cursor.

Preferred Qualifications

• Deep expertise in the AWS ecosystem, including Glue, Athena, SageMaker, and EMR.
• Advanced experience with Data Warehouse technologies, especially Apache Iceberg or BigQuery.
• Leadership or mentorship experience within a data engineering or analytics unit.
• Background or domain knowledge in cybersecurity is a strong advantage.

Our Commitment

We’re trailblazers that dream big, take risks, and challenge cybersecurity’s status quo. It’s simple: we can’t accomplish our mission without diverse teams innovating, together.

We are committed to providing reasonable accommodations for all qualified individuals with a disability. If you require assistance or accommodation due to a disability or special need, please contact us at [e-mail].

Palo Alto Networks is an equal opportunity employer. We celebrate diversity in our workplace, and all qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or other legally protected characteristics.

All your information will be kept confidential according to EEO guidelines.

Is role eligible for Immigration Sponsorship? No. Please note that we will not sponsor applicants for work visas for this position.