GCP Data Engineer: Scale Petabyte Pipelines

SmartRecruiters, Inc.; Sydney; On-site; AUD 120,000 - 180,000; Full time

Job summary

NCS Australia is seeking a GCP Data Engineer to design, build, and scale massive data pipelines using BigQuery, Dataflow, and other GCP tools. You will translate complex requirements into robust, fault-tolerant data solutions while focusing on performance, security, and reliability.

You will collaborate with product managers and data scientists to enable data-driven decisions, ML model training, and operational efficiency at scale across the organization.

Qualifications

• Strong experience with Google Cloud Platform and data engineering.
• Experience with real-time streaming data processing (Pub/Sub, Dataflow Streaming).
• Advanced knowledge of data architectures, data quality, and secure, scalable pipelines.

Responsibilities

• Design and build scalable data pipelines (ETL/ELT) on GCP.
• Architect and optimize BigQuery data models for performance and cost.
• Collaborate with data scientists, analysts, and engineers to ensure data quality and security.
• Monitor and troubleshoot production data systems with CI/CD for data pipelines.
• Explore new cloud data technologies to enhance architecture and governance.

Skills

GCP experience

Data engineering

Streaming data

Python scripting

CI/CD practices

Security best practices

Tools

BigQuery

Dataflow

Composer

Pub/Sub

Terraform

Kubernetes

Docker

Python

CI/CD tooling

Networking (VPCs)

Job description

NCS Australia is seeking a GCP Data Engineer to design, build, and scale massive data pipelines using BigQuery, Dataflow, and other GCP tools. You will translate complex requirements into robust, fault-tolerant data solutions while focusing on performance, security, and reliability.

You will collaborate with product managers and data scientists to enable data-driven decisions, ML model training, and operational efficiency at scale across the organization.