Senior Data Engineer (Hybrid) - Climate & Resources

The Very Group; Parramatta; Hybrid; AUD 133,000 - 147,000; Full time

Job summary

NSW Department of Climate Change, Energy, the Environment and Water (DCCEEW) is seeking a Senior Data Engineer to shape its modern data environment. You will design, develop and support trusted data products used across climate, energy and water management.

You will collaborate with analysts, engineers and domain experts to turn diverse data sources into reliable, scalable solutions, building platform governance and automation to ensure data products remain available and secure.

Qualifications

• Extensive experience in data engineering including design, development and production support.

Responsibilities

• Build robust transformation processes in Fabric Notebooks using modular Python and PySpark code.
• Apply Medallion architectures to create well-structured lakehouses and warehouses.
• Develop data products that are reliable, observable and fit for purpose with strong data quality controls.
• Monitor pipeline performance and resolve issues before they affect downstream use.
• Collaborate with domain experts to translate data challenges into mesh-aligned solutions.
• Share knowledge through reusable notebooks, templates and engineering practices.
• Operate the enterprise data platform including monitoring, access management and incident resolution.
• Improve governance, controls, monitoring and automation for data products.

Skills

Python

PySpark

SQL

Medallion architecture

Delta Lake

Microsoft Fabric

Data quality practices

Agile teamwork

Notebooks

Data governance

Tools

Microsoft Fabric

Delta Lake

Job description

NSW Department of Climate Change, Energy, the Environment and Water (DCCEEW) is seeking a Senior Data Engineer to shape its modern data environment. You will design, develop and support trusted data products used across climate, energy and water management.

You will collaborate with analysts, engineers and domain experts to turn diverse data sources into reliable, scalable solutions, building platform governance and automation to ensure data products remain available and secure.