Senior Data Engineer

Sharp & Carter; City of Melbourne; On-site; AUD 140,000 - 180,000; Full time

Job summary

Sharp & Carter is seeking a Senior Data Engineer to advance a Data Strategy by building and managing a Microsoft Fabric data platform in Melbourne.

You will act as the technical SME for Microsoft Fabric, designing, building and maintaining secure end-to-end data solutions from source connectivity through transformation, storage and semantic models for reporting and analytics.

Qualifications

• Hands-on experience designing and delivering production solutions using Microsoft Fabric.
• Experience delivering end-to-end data solutions with secure connectivity, ingestion, transformation, storage and orchestration.
• Experience using Fabric capabilities: Data Factory pipelines, Copy Job, Dataflow Gen2, Notebooks, Lakehouse and Warehouse.
• Strong SQL skills and Python/PySpark for data engineering and transformation.
• Strong data modelling including PowerBI semantic models.

Responsibilities

• Act as the Microsoft Fabric technical SME, providing technical leadership around architecture, tools and implementation.
• Build secure connectivity between Microsoft Fabric and source systems.
• Design/manage data ingestion and integration using Fabric.
• Develop data pipelines, transformations, Lakehouse and Warehouse solutions.
• Build trusted data models and Power BI semantic models.
• Own the end-to-end data engineering lifecycle: monitoring, troubleshooting, optimisation and deployment.
• Establish development, security, version control and CI/CD practices.
• Work directly with business and technical stakeholders to turn requirements into secure, scalable solutions and produce technical documentation.

Skills

Microsoft Fabric

SQL

Python/PySpark

Power BI

Data modelling

Tools

Data Factory pipelines

Copy Job

Dataflow Gen2

Notebooks

Lakehouse

Warehouse

Job description

We are seeking a Senior Data Engineer to support our client's Data Strategy by developing and managing its Microsoft Fabric data platform.

You will act as the technical SME for Microsoft Fabric and independently design, build and maintain secure end to end data solutions from source connectivity and ingestion through transformation, storage and semantic models for reporting/analytics.

The core responsibilities are:

• Act as the Microsoft Fabric technical SME, providing technical leadership around architecture, tools and implementation.
• Build secure connectivity between Microsoft Fabric and source systems.
• Design/manage data ingestion and integration using Fabric.
• Develop data pipelines, transformations, Lakehouse and Warehouse solutions.
• Build trusted data models and Power BI semantic models.
• Own the end-to-end data engineering lifecycle: monitoring, troubleshooting, optimisation and deployment.
• Establish development, security, version control and CI/CD practices.
• Work directly with business and technical stakeholders to turn requirements into secure, scalable solutions and produce technical documentation.

To be considered, you must demonstrate the following:

• Demonstrated hands-on experience designing and delivering production solutions using Microsoft Fabric, with ability to operate independently from commencement.
• Demonstrated experience delivering end-to-end data solutions, including secure source connectivity, ingestion, transformation, storage and orchestration.
• Strong practical experience selecting and using appropriate Microsoft Fabric capabilities, including Data Factory pipelines, Copy Job, Dataflow Gen2, Notebooks, Lakehouse and Warehouse.
• Strong SQL skills and practical experience using Python/PySpark for data engineering and transformation workloads.
• Strong data modelling experience including development of PowerBI semantic models