Senior Data Engineer - Hybrid, Climate Data Platform

NSW Department of Climate Change, Energy, the Environment and Water; Sydney; Hybrid; AUD 133,000 - 147,000; Full time

Job summary

The NSW Department of Climate Change, Energy, the Environment and Water is seeking a Senior Data Engineer to shape the department’s modern data environment and build trusted data products for climate, energy and water management.

You will design, develop and operate pipelines, apply Medallion architecture, and work with Fabric, PySpark, Python, and SQL. Hybrid work with Parramatta preferred to support data-driven decision making.

Qualifications

• Extensive data engineering experience delivering production data solutions.
• Practical experience with Microsoft Fabric including Lakehouses, Warehouses, Data Pipeline, Mirroring and Notebooks.
• Strong programming skills in Python, PySpark and SQL.
• Experience developing scalable ETL/ELT processes with maintainable code.
• Knowledge of Delta Lake, schema evolution and data quality practices.
• Experience in monitoring, auditing, error handling and operational controls for data pipelines.
• Strong understanding of Microsoft Azure and Azure DevOps.
• Ability to work in agile teams and translate business data needs into data products.

Responsibilities

• Design, develop and support data products used across the department.
• Build robust ingestion pipelines and data transformations in Fabric Notebooks.
• Apply Medallion architecture patterns to create well-structured data lakehouses and warehouses.
• Ensure data products are observable, auditable and of high quality.
• Monitor pipeline performance and resolve issues proactively.
• Collaborate with domain experts to translate data challenges into solutions.
• Share knowledge via reusable notebooks, templates and engineering practices.

Skills

Python

PySpark

SQL

Microsoft Fabric

Azure DevOps

Data pipelines

Medallion architecture

Tools

Microsoft Fabric

Delta Lake

Azure

Azure DevOps

Job description

The NSW Department of Climate Change, Energy, the Environment and Water is seeking a Senior Data Engineer to shape the department’s modern data environment and build trusted data products for climate, energy and water management.

You will design, develop and operate pipelines, apply Medallion architecture, and work with Fabric, PySpark, Python, and SQL. Hybrid work with Parramatta preferred to support data-driven decision making.