Databricks Data Engineer – SQL/ETL Migration Lead

Talent; City of Melbourne; On-site; AUD 140,000 - 200,000; Full time

Benefits offered by this job

Contract until end of April 2027

Government Agency

Job summary

Talent in Melbourne is seeking an experienced Data Engineer to lead a major migration project from SQL Server to Databricks on Azure. You will design and implement scalable data pipelines, notebooks, and ETL workflows using SQL and Python, and ensure data quality across the migration.

The role requires 10+ years in data engineering, strong Databricks experience, advanced SQL skills, and a track record migrating data warehouses to modern platforms.

Qualifications

• 10+ years of hands-on experience as a Data Engineer.
• Advanced SQL skills with proven experience working with data warehouses.
• Strong Python development skills for building data pipelines.
• Proven experience migrating SQL Server data warehouses, ETL processes or data platforms to modern data platforms.

Responsibilities

• Lead the migration of an existing SQL Server data warehouse and ETL environment into Databricks and Azure.
• Translate existing SQL warehouse structures, queries, stored procedures and ETL logic into scalable Databricks solutions.
• Build and develop Databricks notebooks, data pipelines and workflows using SQL and Python.
• Analyse existing SQL data and processes, identify migration requirements.
• Support data validation, testing, optimisation and troubleshooting throughout the migration.

Tools

Databricks

Azure

Python

SQL

Job description

Talent in Melbourne is seeking an experienced Data Engineer to lead a major migration project from SQL Server to Databricks on Azure. You will design and implement scalable data pipelines, notebooks, and ETL workflows using SQL and Python, and ensure data quality across the migration.

The role requires 10+ years in data engineering, strong Databricks experience, advanced SQL skills, and a track record migrating data warehouses to modern platforms.