Senior Data Engineer

Agrifutures; City of Melbourne; On-site; AUD 120,000 - 170,000; Full time

Job summary

Agrifutures is seeking a Senior Data Engineer to advance our Data Strategy by building and managing a Microsoft Fabric data platform. You will act as the technical SME for Fabric, independently designing secure end-to-end data solutions from source connectivity to semantic models for reporting and analytics.

The role covers architecture leadership, secure data connectivity, data ingestion and transformation, Lakehouse and Warehouse solutions, and Power BI semantic modeling, with ownership of the

Qualifications

• Hands-on experience designing and delivering production solutions with Microsoft Fabric.
• End-to-end data solutions including secure source connectivity, ingestion, transformation and storage.
• Experience with Fabric capabilities: Data Factory pipelines, Copy Job, Dataflow Gen2, Notebooks, Lakehouse, Warehouse.
• Strong SQL skills and Python/PySpark for data engineering.
• Experience in developing Power BI semantic models.

Responsibilities

• Act as Microsoft Fabric SME and provide architectural leadership.
• Build secure connectivity between Fabric and source systems.
• Design and manage data ingestion and integration using Fabric.
• Develop data pipelines, transformations and storage solutions.
• Create trusted data and Power BI semantic models.
• Own end-to-end data engineering lifecycle: monitoring, troubleshooting, deployment.
• Establish development, security, version control and CI/CD practices.
• Collaborate with stakeholders to translate requirements into secure, scalable solutions and document them.

Skills

Microsoft Fabric

SQL

Python/PySpark

Power BI

Data modeling

Data pipelines

Data governance

CI/CD

Notebooks

Lakehouse/Warehouse

Tools

Data Factory pipelines

Copy Job

Dataflow Gen2

Job description

We are seeking a Senior Data Engineer to support our client’s Data Strategy by developing and managing its Microsoft Fabric data platform.

You will act as the technical SME for Microsoft Fabric and independently design, build and maintain secure end to end data solutions from source connectivity and ingestion through transformation, storage and semantic models for reporting/analytics.

The core responsibilities are:

• Act as the Microsoft Fabric technical SME, providing technical leadership around architecture, tools and implementation.
• Build secure connectivity between Microsoft Fabric and source systems.
• Design/manage data ingestion and integration using Fabric.
• Develop data pipelines, transformations, Lakehouse and Warehouse solutions.
• Build trusted data models and Power BI semantic models.
• Own the end-to-end data engineering lifecycle: monitoring, troubleshooting, optimisation and deployment.
• Establish development, security, version control and CI/CD practices.
• Work directly with business and technical stakeholders to turn requirements into secure, scalable solutions and produce technical documentation.

To be considered, you must demonstrate the following:

• Demonstrated hands-on experience designing and delivering production solutions using Microsoft Fabric, with ability to operate independently from commencement.
• Demonstrated experience delivering end-to-end data solutions, including secure source connectivity, ingestion, transformation, storage and orchestration.
• Strong practical experience selecting and using appropriate Microsoft Fabric capabilities, including Data Factory pipelines, Copy Job, Dataflow Gen2, Notebooks, Lakehouse and Warehouse.
• Strong SQL skills and practical experience using Python/PySpark for data engineering and transformation workloads.
• Strong data modelling experience including development of PowerBI semantic models