Data Engineer - GCP Pipelines & BigQuery

XPT Software Australia Pty Ltd; Ryde; Presencial; AUD 110.000 - 150.000; Jornada completa

Descripción de la vacante

XPT Software Australia Pty Ltd is seeking a data engineering professional to design, develop, and maintain scalable data pipelines on Google Cloud Platform. You will implement ETL/ELT for batch and real-time processing using PySpark, Apache Beam, Dataflow, and Pub/Sub, and work with BigQuery for warehousing and SQL optimisation.

The role involves building data lakes and warehouses on Google Cloud, implementing data quality measures, and collaborating with cross-functional teams to deliver robust

Responsabilidades

• Design, develop, and maintain scalable data pipelines on GCP.
• Build and optimise batch and real-time ETL/ELT workflows.
• Develop data processing solutions with Python, PySpark and Apache Beam.

Conocimientos

GCP services

ETL/ELT

BigQuery

SQL development

Python

PySpark

Apache Beam

Cloud Dataflow

Pub/Sub

Cloud Composer

Descripción del empleo

XPT Software Australia Pty Ltd is seeking a data engineering professional to design, develop, and maintain scalable data pipelines on Google Cloud Platform. You will implement ETL/ELT for batch and real-time processing using PySpark, Apache Beam, Dataflow, and Pub/Sub, and work with BigQuery for warehousing and SQL optimisation.

The role involves building data lakes and warehouses on Google Cloud, implementing data quality measures, and collaborating with cross-functional teams to deliver robust