GCP Principal Data Engineer — Scale Data Pipelines

NCS Australia; Sydney; On-site; AUD 150,000 - 190,000; Full time

Job summary

NCS Australia is seeking a seasoned GCP Data Engineer to design and scale our data infrastructure. You will build end-to-end data pipelines and work with BigQuery, Dataflow, Cloud Composer and Pub/Sub to enable data-driven decisions across our products.

Collaborate with data scientists, analysts and software engineers to ensure data quality, security and throughput. Tackle petabyte-scale data challenges with robust, scalable solutions in a fast-moving environment.

Qualifications

• Strong background in Google Cloud Platform (GCP) and experience as a GCP Data Engineer.
• Experience with real-time streaming data processing using Pub/Sub and Dataflow.
• Familiarity with containerization/orchestration (Docker, Kubernetes).
• Experience with version control (Git) and DataOps/data quality practices.

Responsibilities

• Design, build, and maintain scalable data pipelines (ETL/ELT) on GCP.
• Architect and optimize BigQuery data models for performance and cost.
• Collaborate with data scientists, analysts, and software engineers to ensure data quality and security.
• Monitor and improve CI/CD for data pipelines and production data systems.
• Explore new cloud services to enhance data architecture and governance.

Skills

GCP data engineer

Pub/Sub

BigQuery

Dataflow/Beam

Kubernetes

Terraform

Python

CI/CD

Networking basics

Cloud security

Cloud Architect/DevOps certs

Tools

Docker

Git

Cloud Composer

Terraform

Job description

NCS Australia is seeking a seasoned GCP Data Engineer to design and scale our data infrastructure. You will build end-to-end data pipelines and work with BigQuery, Dataflow, Cloud Composer and Pub/Sub to enable data-driven decisions across our products.

Collaborate with data scientists, analysts and software engineers to ensure data quality, security and throughput. Tackle petabyte-scale data challenges with robust, scalable solutions in a fast-moving environment.