GCP Data Engineer - Scalable Pipelines & BigQuery

NCS Group; Sydney; On-site; AUD 140,000 - 190,000; Full time

Job summary

NCS Australia is seeking a GCP Data Engineer to design and scale data pipelines, transforming raw, high-volume data into structured insights used across the business. You will build end-to-end data processing systems on Google Cloud, focusing on robustness, security, and performance with BigQuery, Dataflow, Pub/Sub, and Cloud Composer.

Collaborate with data scientists, analysts and engineers to meet data needs, implement CI/CD for pipelines, and explore new cloud technologies to improve

Qualifications

• Strong background in Google Cloud Platform and a Data Engineer role.
• Experience with real-time streaming using Pub/Sub and Dataflow Streaming.
• Familiarity with containerization and orchestration tools like Docker and Kubernetes.
• Experience with version control systems (Git).
• Deep understanding of DataOps and automated data quality testing.
• Previous experience delivering secure Google Cloud solutions at an enterprise level.
• Advanced Terraform for infrastructure as code.
• Scripting in Python or Bash.
• Knowledge of CI/CD for data platforms.
• Experience with core GCP services: Compute Engine, App Engine, Kubernetes Engine, Cloud Storage, BigQuery, Pub/Sub.

Responsibilities

• Design, build, and maintain scalable data pipelines (ETL/ELT).
• Architect and optimize BigQuery data models for performance and cost.
• Collaborate with Data Scientists, Analysts, and Engineers to ensure data quality and security.
• Monitor, troubleshoot, and improve production data systems and CI/CD pipelines.
• Explore new cloud technologies to enhance data architecture and governance.

Skills

GCP Data Engineer

DataOps

CI/CD

Python

Shell scripting

Tools

Dataflow (Apache Beam)

BigQuery

Pub/Sub

Cloud Composer (Airflow)

Terraform

Docker

Kubernetes

Git

Job description

NCS Australia is seeking a GCP Data Engineer to design and scale data pipelines, transforming raw, high-volume data into structured insights used across the business. You will build end-to-end data processing systems on Google Cloud, focusing on robustness, security, and performance with BigQuery, Dataflow, Pub/Sub, and Cloud Composer.

Collaborate with data scientists, analysts and engineers to meet data needs, implement CI/CD for pipelines, and explore new cloud technologies to improve