Data engineers

Exclaim IT; Canberra; On-site; AUD 120,000 - 180,000; Full time

Job summary

Exclaim IT is seeking an experienced Data Engineer with strong Google Cloud Platform (GCP) expertise to design, develop and maintain scalable data solutions within a complex enterprise environment.

Please note: A current NV1 security clearance is mandatory for this position. Location of work includes multiple states across Australia.

Qualifications

• Minimum three years of data engineering experience, including at least two years designing, building and operating production data solutions on Google Cloud Platform (GCP).
• Proven track record delivering and supporting production-grade data pipelines in complex enterprise environments.
• Strong hands-on experience with BigQuery, including data modelling, partitioning, clustering, query optimisation, cost management and workload monitoring.
• Experience developing scalable batch and streaming data pipelines using Dataflow and Apache Beam.
• Demonstrated experience applying data warehousing, dimensional modelling and data quality practices.
• Strong proficiency in SQL and Python for developing, testing and optimising data pipelines.
• Experience implementing DataOps practices, including CI/CD, infrastructure as code, automated testing and monitoring.
• Hands-on experience with Terraform, Git-based source control and CI/CD tooling such as Cloud Build or GitHub Actions.
• Demonstrated understanding of cloud security, data governance, metadata management and data lifecycle principles.
• Ability to translate business and technical requirements into scalable, secure and maintainable data solutions.

Responsibilities

• Design, develop and maintain scalable batch, micro-batch and real-time data ingestion, transformation and curation pipelines.
• Translate solution designs, technical specifications and data contracts into robust, production-ready data solutions.
• Apply data warehousing, dimensional modelling, data quality and governance principles, including metadata, lineage, classification, retention and security.
• Develop and optimise data pipelines using GCP services, including BigQuery, Dataflow, Apache Beam, Cloud Storage, Cloud Composer and Pub/Sub.
• Implement DataOps practices, including CI/CD, infrastructure automation, automated testing, monitoring, alerting and release management.
• Identify and address gaps between solution designs and technical implementation, ensuring requirements are fully realised.
• Optimise data solutions for performance, scalability, reliability, cost efficiency and operational supportability.
• Apply security best practices, including least-privilege access, encryption, secrets management and secure handling of sensitive information.
• Build and maintain automated unit, integration, reconciliation and data quality tests for data pipelines and products.
• Collaborate with technical teams and business stakeholders throughout the solution delivery lifecycle.
• Identify technical risks, dependencies and design deficiencies, recommending sustainable improvements.
• Contribute to technical analysis, estimation, solution planning, standardisation and continuous improvement initiatives.
• Develop reusable data products and trusted datasets to support advanced analytics, machine learning and generative AI.

Skills

GCP Data Eng

SQL

Python

Data modeling

Data governance

Security best practices

CI/CD

Stakeholder collab

Tools

BigQuery

Dataflow

Cloud Composer

Pub/Sub

Terraform

GitHub Actions

Job description

About the role

We are seeking an experienced Data Engineer with strong Google Cloud Platform (GCP) expertise to design, develop and maintain scalable, secure and reliable data solutions within a complex enterprise environment.

Please note: A current Negative Vetting Level 1 (NV1) security clearance is mandatory for this position.

Location of work : QLD, WA, ACT, VIC, NSW, NT, SA and TAS.

Key responsibilities

• Design, develop and maintain scalable batch, micro-batch and real-time data ingestion, transformation and curation pipelines.
• Translate solution designs, technical specifications and data contracts into robust, production-ready data solutions.
• Apply data warehousing, dimensional modelling, data quality and governance principles, including metadata, lineage, classification, retention and security.
• Develop and optimise data pipelines using GCP services, including BigQuery, Dataflow, Apache Beam, Cloud Storage, Cloud Composer and Pub/Sub.
• Implement DataOps practices, including CI/CD, infrastructure automation, automated testing, monitoring, alerting and release management.
• Identify and address gaps between solution designs and technical implementation, ensuring requirements are fully realised.
• Optimise data solutions for performance, scalability, reliability, cost efficiency and operational supportability.
• Apply security best practices, including least-privilege access, encryption, secrets management and secure handling of sensitive information.
• Build and maintain automated unit, integration, reconciliation and data quality tests for data pipelines and products.
• Collaborate with technical teams and business stakeholders throughout the solution delivery lifecycle.
• Identify technical risks, dependencies and design deficiencies, recommending sustainable improvements.
• Contribute to technical analysis, estimation, solution planning, standardisation and continuous improvement initiatives.
• Develop reusable data products and trusted datasets to support advanced analytics, machine learning and generative AI.

Highly desirable criteria

• Minimum three years of data engineering experience, including at least two years designing, building and operating production data solutions on Google Cloud Platform (GCP).
• Demonstrated experience delivering and supporting production-grade data pipelines in complex enterprise environments.
• Strong hands-on experience with BigQuery, including data modelling, partitioning, clustering, query optimisation, cost management and workload monitoring.
• Experience developing scalable batch and streaming data pipelines using Dataflow and Apache Beam.
• Demonstrated experience applying data warehousing, dimensional modelling and data quality practices.
• Strong proficiency in SQL and Python for developing, testing and optimising data pipelines.
• Experience implementing DataOps practices, including CI/CD, infrastructure as code, automated testing and monitoring.
• Hands-on experience with Terraform, Git-based source control and CI/CD tooling such as Cloud Build or GitHub Actions.
• Demonstrated understanding of cloud security, data governance, metadata management and data lifecycle principles.
• Ability to translate business and technical requirements into scalable, secure and maintainable data solutions.

Contact details

For a confidential chat, please contact Shruthi on 0474 545 379 or via email at [e-mail]. We would be more than happy to provide you with additional information and assist in the application process.

Please note that a National Police Check might be required as part of pre-employment screening.