Microsoft Fabric Data Engineer

BULLIT RECRUITMENT SOLUTIONS LIMITED; Sydney; On-site; AUD 140,000 - 190,000; Full time

Job summary

BULLIT RECRUITMENT SOLUTIONS LIMITED seeks a Senior Data Engineer to design and build end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory. You will manage Lakehouse and OneLake architectures for diverse data, ingest data from databases, APIs, and files, and implement ETL/ELT with PySpark and SQL.

You will optimize data models for analytics, build Fabric Data Warehouse solutions with T-SQL, ensure data quality and security, and enable Power BI semantic models for

Qualifications

• Design enterprise data architecture leveraging Fabric components (Lakehouse, Warehouse, Dataflows Gen2, Pipelines, OneLake).
• Develop reusable ETL/ELT frameworks using PySpark, SQL, and Fabric notebooks.
• Manage CI/CD workflows via Azure DevOps and Git.
• Architect scalable, cost-efficient data solutions integrating Fabric with Azure/enterprise systems.
• Apply best practices for performance tuning, cost optimisation, and data quality.

Responsibilities

• Design end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory.
• Develop and manage Lakehouse and OneLake architectures for structured and unstructured data.
• Ingest data from multiple sources into Fabric.
• Implement ETL/ELT processes using Spark (PySpark) and SQL.
• Design and optimize data models to support analytics and reporting workloads.
• Develop and maintain Fabric Data Warehouse solutions using T-SQL.
• Ensure data quality, reliability, and consistency through validation and monitoring.
• Optimize pipeline performance, storage, and capacity utilization in Fabric.
• Implement security, access control (RBAC), and data sensitivity labels.
• Enable and support Power BI semantic models and downstream reporting use cases.
• Monitor, troubleshoot, and resolve data pipeline and performance issues.
• Implement data governance, lineage, and cataloging using Microsoft Purview.
• Support CI/CD and DevOps practices for analytics and data engineering workloads.
• Collaborate with architects, analysts, and stakeholders to translate business requirements into scalable data solutions.

Skills

Data architecture

ETL/ELT design

PySpark

SQL

Data modeling

Security & RBAC

Power BI integration

CI/CD

DevOps for analytics

Leadership

Education

Bachelor's degree in CS/IS or equivalent

DP-203 or similar certification

Fabric Analytics Engineer certification (preferred)

Tools

Azure DevOps

Git

Microsoft Fabric

Purview

Delta Lake

ADLS

OneLake

Power BI

Job description

• Design, build, and maintain end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory.
• Develop and manage Lakehouse and OneLake architectures for structured and unstructured data.
• Ingest data from multiple sources (databases, APIs, files, streaming sources) into Fabric.
• Implement ETL/ELT processes using Spark (PySpark) and SQL.
• Design and optimize data models to support analytics and reporting workloads.
• Develop and maintain Fabric Data Warehouse solutions using T-SQL.
• Ensure data quality, reliability, and consistency through validation and monitoring.
• Optimize pipeline performance, storage, and capacity utilization in Fabric.
• Implement security, access control (RBAC), and data sensitivity labels.
• Enable and support Power BI semantic models and downstream reporting use cases.
• Monitor, troubleshoot, and resolve data pipeline and performance issues.
• Implement data governance, lineage, and cataloging using Microsoft Purview.
• Support CI/CD and DevOps practices for analytics and data engineering workloads.
• Collaborate with architects, analysts, and stakeholders to translate business requirements into scalable data solutions.

Skill Experience requirement :

• Design enterprise data architecture leveraging Fabric components (Lakehouse, Warehouse, Dataflows Gen2, Pipelines, OneLake).
• Build and optimise Fabric Lakehouse tables, Warehouses, Pipelines, and Dataflows Gen2.
• Define data modelling standards, governance frameworks, and security patterns.
• Develop reusable ETL/ELT frameworks using PySpark, SQL, and Fabric notebooks.
• Manage CI/CD workflows via Azure DevOps and Git.
• Architect scalable, cost-efficient data solutions integrating Fabric with Azure/enterprise systems.
• Apply best practices for performance tuning, cost optimisation, and data quality.
• Provide technical leadership and guidance to engineering teams.
• Expertise in Delta Lake, Fabric Data Engineering, ADLS, PySpark/SQL.
• Strong understanding of data modelling and integration with Power BI datasets.
• Strong understanding of metadata management, Purview, access control, and FinOps.
• Certifications: DP-203, Fabric Analytics Engineer (preferred).

• Design, build, and maintain end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory.
• Develop and manage Lakehouse and OneLake architectures for structured and unstructured data.
• Ingest data from multiple sources (databases, APIs, files, streaming sources) into Fabric.
• Implement ETL/ELT processes using Spark (PySpark) and SQL.
• Design and optimize data models to support analytics and reporting workloads.
• Develop and maintain Fabric Data Warehouse solutions using T-SQL.
• Ensure data quality, reliability, and consistency through validation and monitoring.
• Optimize pipeline performance, storage, and capacity utilization in Fabric.
• Implement security, access control (RBAC), and data sensitivity labels.
• Enable and support Power BI semantic models and downstream reporting use cases.
• Monitor, troubleshoot, and resolve data pipeline and performance issues.
• Implement data governance, lineage, and cataloging using Microsoft Purview.
• Support CI/CD and DevOps practices for analytics and data engineering workloads.
• Collaborate with architects, analysts, and stakeholders to translate business requirements into scalable data solutions.