Senior Data Engineer

Talent; City of Melbourne; On-site; AUD 140,000 - 200,000; Full time

Benefits offered by this job

Contract until end of April 2027

Government Agency

Job summary

Talent in Melbourne is seeking an experienced Data Engineer to lead a major migration project from SQL Server to Databricks on Azure. You will design and implement scalable data pipelines, notebooks, and ETL workflows using SQL and Python, and ensure data quality across the migration.

The role requires 10+ years in data engineering, strong Databricks experience, advanced SQL skills, and a track record migrating data warehouses to modern platforms.

Qualifications

• 10+ years of hands-on experience as a Data Engineer.
• Advanced SQL skills with proven experience working with data warehouses.
• Strong Python development skills for building data pipelines.
• Proven experience migrating SQL Server data warehouses, ETL processes or data platforms to modern data platforms.

Responsibilities

• Lead the migration of an existing SQL Server data warehouse and ETL environment into Databricks and Azure.
• Translate existing SQL warehouse structures, queries, stored procedures and ETL logic into scalable Databricks solutions.
• Build and develop Databricks notebooks, data pipelines and workflows using SQL and Python.
• Analyse existing SQL data and processes, identify migration requirements.
• Support data validation, testing, optimisation and troubleshooting throughout the migration.

Tools

Databricks

Azure

Python

SQL

Job description

• Contract until end of April 2027 – Government Agency
• Databricks, Python, SQL and Azure

We are seeking a Data Engineer – Databricks to join a major data platform migration program. You will play a hands‑on role in transitioning an existing SQL Server data warehouse and associated pipelines to a modern Databricks and Azure environment.

Key Responsibilities

Lead the migration of an existing SQL Server data warehouse and ETL environment into Databricks and Azure.

• Translate existing SQL warehouse structures, queries, stored procedures and ETL logic into scalable Databricks solutions.
• Build and develop Databricks notebooks, data pipelines and workflows using SQL and Python.
• Analyse existing SQL data and processes, identify migration requirements
• Support data validation, testing, optimisation and troubleshooting throughout the migration

Key Skills and Experience

• 10+ years of hands‑on experience as a Data Engineer.
• Strong hands‑on experience with Databricks, including data engineering, notebooks, workflows and pipelines.
• Advanced SQL skills, with proven experience working with data warehouses
• Strong Python development skills, particularly for building data pipelines
• Proven experience migrating SQL Server data warehouses, ETL processes or data platforms to Databricks or similar modern data platforms.

What’s in it for you:

• Contract until end of April 2027
• Government Agency