Lead Azure ETL Engineer

The Recruitment Hive Pty Ltd; Canberra; On-site; AUD 120,000 - 180,000; Full time

Job summary

The Recruitment Hive Pty Ltd is seeking a Lead Azure ETL Engineer to drive end-to-end data movement, transformation and visualisation for a Federal Government department in Canberra. The role focuses on Azure Data Factory, Databricks, SQL Server and Data Lake Storage, with DevOps and Visual Studio for development.

The ideal candidate will design scalable ETL/ELT processes, optimise cloud data for analytics, and build impactful Power BI visuals for senior stakeholders.

Qualifications

• Extensive experience building ETL/ELT for large data movement and visualisation.
• Experience optimizing data for cloud analytics engines (Spark, Azure SQL).
• Proven collaboration with engineering, storage and analytics teams.

Responsibilities

• Lead ETL/ELT design for Azure cloud environments.
• Develop data pipelines using Data Factory, Databricks and SQL Server.
• Create tailored visualisations with Power BI.
• Ensure data quality and performance for analytics.
• Collaborate with cross-functional teams to meet business needs.

Skills

ETL/ELT design

Cloud data integration

Azure data services

Power BI visualization

Tools

Azure Data Factory

Databricks

SQL Server

Data Lake Storage

DevOps

Visual Studio

Spark

Job description

• ETL/ELT for large data movement, transformation and visualisations


• NV1 Clearance - 12 + 12 + 12 month contract

Our client, a Federal Government department based in Civic, ACT, that builds stronger and more sustainable agriculture, fisheries, forestry and land care, requires the services of a Lead Azure ETL Engineer for a 12 + 12 + 12 month contract.

Duties of the Lead Azure ETL Engineer:

• Azure hosted services including: Data Integration - Data Factory, SQL Server Integration Services and/or Databricks.


• Data Store - SQL Server and/or Data Lake Storage.


• Development tools – DevOps, Visual Studio.


• Data technology solutions – sourcing (Oracle, Ingres, Azure, SQL Server), automated ingestion.


• Transformation of data into formats tailored for advanced analytics and AI use cases – Parquet and/or Delta.


• Analyse and interpret complex data sets, and to identify trends and patterns.


• Design principles to create appropriate visualisations for target audience.


• Visualisation tools - Power BI.

Skills and Experience - Lead Azure ETL Engineer:

• Demonstrated experience developing ETL/ELT processes for complex and/or large data movement, transformation and/or visualisations, particularly in a cloud environment.


• Experience preparing data optimised for query performance in cloud computed engines. E.g. Distributed computing engines (Spark), Graph Databases & Azure SQL.


• Experience working with Engineering, Storage and Analytics services in cloud infrastructure.


• Experience working with Azure Data Factory and Databricks.


• Experience/Knowledge of working with Data Lake and Lakehouses.

We Value Diversity

The Recruitment Hive is a proud Equal Opportunity Employer. We recognise the value of diverse perspectives and experience and are committed to an inclusive workplace where everyone feels valued and respected for who they are. We encourage people with disability, indigenous, veterans and people of all backgrounds and abilities to express their interest in this role.