Senior Cloud Data Platform Engineer

Innovate It Australia; Sydney; On-site; AUD 140,000 - 190,000; Full time

Benefits offered by this job

Paid parental leave

Well-being initiatives

Discounts on products/services

Recognition programs

AI-driven learning platform

Job summary

NCS Australia is seeking a Data Engineer to design, build, and scale massive data pipelines across the organization. You will translate complex requirements into robust, fault-tolerant solutions on Google Cloud Platform, focusing on robustness, efficiency, security, and high throughput.

You will collaborate with data scientists, analysts, and software engineers to enable data-driven decision making, ML model training, and operational efficiency, supporting petabyte-scale data challenges and

Qualifications

• Experience designing and building scalable data pipelines (ETL/ELT) using GCP services.
• Strong background in Google Cloud Platform and data engineering.
• Experience with real-time streaming using Pub/Sub and Dataflow Streaming.
• Familiarity with containers and orchestration tools (Docker, Kubernetes).
• Proficient with version control (Git) and DataOps principles.

Responsibilities

• Design and build scalable data pipelines (ETL/ELT) using Dataflow (Apache Beam), Composer (Apache Airflow), and BigQuery.
• Architect and optimize data models within BigQuery for performance and cost efficiency, supporting high-volume analytical workloads.
• Collaborate with Data Scientists, Analysts, and Software Engineers to understand data needs and ensure data quality, integrity, and security across platforms.
• Monitor and troubleshoot production data systems, implementing CI/CD practices for data pipelines.
• Explore and recommend new data technologies and cloud services to enhance data architecture and governance.

Skills

Data pipelines

BigQuery

Dataflow

Apache Airflow

Pub/Sub

Python

Bash

Terraform

Docker

Kubernetes

GCP

CI/CD

DataOps

Tools

Git

Terraform

Docker

Kubernetes

Job description

NCS Australia is seeking a Data Engineer to design, build, and scale massive data pipelines across the organization. You will translate complex requirements into robust, fault-tolerant solutions on Google Cloud Platform, focusing on robustness, efficiency, security, and high throughput.

You will collaborate with data scientists, analysts, and software engineers to enable data-driven decision making, ML model training, and operational efficiency, supporting petabyte-scale data challenges and