Lead Data Engineer - Hybrid (Canberra, Security Clearance)

IT Alliance Australia; Canberra; Hybrid; AUD 170,000 - 230,000; Full time

Job summary

IT Alliance Australia is seeking a Lead Data Engineer for a Federal Government client in Canberra. This long-term contract offers a challenging role leading data engineering efforts across ingestion, transformation, and analytics.

The position requires hands-on ETL/ELT, Spark, data pipelines, and experience with Parquet/Delta, Oracle, Ingres, Azure and SQL Server. A baseline security clearance or higher is required.

Qualifications

• Strong hands-on ETL/ELT development for large-scale data ingestion and transformation.

Responsibilities

• Lead hands-on ETL/ELT development for large-scale data ingestion and transformation.
• Optimise Spark-based workflows for cloud query performance.
• Build automated data ingestion and transformation pipelines.
• Prepare data using Parquet and Delta formats.
• Sourcing/integrating data from Oracle, Ingres, Azure and SQL Server.
• Leverage cloud data engineering, storage and analytics services.
• Create Power BI visualisations to meaningfully represent data.
• Work with ADF, Databricks, Data Lake/Lakehouse, Azure ML and ArcGIS Enterprise where possible.

Skills

ETL/ELT

Spark

Data pipelines

Parquet/Delta

Oracle data

Ingres data

Azure data

SQL Server data

Cloud data engineering

Power BI

ADF

Databricks

Data Lake/Lakehouse

Azure Machine Learning

ArcGIS Enterprise

Tools

Azure Data Factory

Databricks

Azure Synapse

Job description

IT Alliance Australia is seeking a Lead Data Engineer for a Federal Government client in Canberra. This long-term contract offers a challenging role leading data engineering efforts across ingestion, transformation, and analytics.

The position requires hands-on ETL/ELT, Spark, data pipelines, and experience with Parquet/Delta, Oracle, Ingres, Azure and SQL Server. A baseline security clearance or higher is required.