Principal Databricks Engineer: Azure Data Migration

Department of Education; Melbourne; On-site; AUD 150,000 - 190,000; Full time

Job summary

Department of Education in Victoria is seeking a Principal Databricks Developer to lead a critical data platform migration. You will transform legacy SQL Server data warehouses and pipelines into Databricks and Azure solutions, applying your SQL and Python expertise to deliver scalable outcomes.

You will work hands-on on migration design, development, and optimisation, ensuring continuity for downstream reporting and analytics during the transition and mentoring the team to uplift Databricks

Qualifications

• Extensive experience migrating data warehouses to Databricks/Azure.
• Strong SQL and Python skills with hands-on delivery focus.
• Experience with legacy Microsoft data and BI platforms (SSIS/SSAS/SSRS).
• Ability to design, develop and optimise migration pipelines and data quality checks.

Responsibilities

• Design Databricks notebooks, jobs and workflows for data ingestion and transformation.
• Migrate SQL Server data warehouse structures and ETL processes to Databricks/Azure.
• Translate SSIS packages and pipelines to Databricks-based solutions, including Python pipelines.

Skills

Python

SQL

Databricks

Tools

Databricks

SQL Server

Azure

SSIS

SSAS

Power BI

Tableau

Job description

Department of Education in Victoria is seeking a Principal Databricks Developer to lead a critical data platform migration. You will transform legacy SQL Server data warehouses and pipelines into Databricks and Azure solutions, applying your SQL and Python expertise to deliver scalable outcomes.

You will work hands-on on migration design, development, and optimisation, ensuring continuity for downstream reporting and analytics during the transition and mentoring the team to uplift Databricks