Data Engineer

IT Alliance Australia; Canberra; Hybrid; AUD 170,000 - 230,000; Full time

Job summary

IT Alliance Australia is seeking a Lead Data Engineer for a Federal Government client in Canberra. This long-term contract offers a challenging role leading data engineering efforts across ingestion, transformation, and analytics.

The position requires hands-on ETL/ELT, Spark, data pipelines, and experience with Parquet/Delta, Oracle, Ingres, Azure and SQL Server. A baseline security clearance or higher is required.

Qualifications

• Strong hands-on ETL/ELT development for large-scale data ingestion and transformation.

Responsibilities

• Lead hands-on ETL/ELT development for large-scale data ingestion and transformation.
• Optimise Spark-based workflows for cloud query performance.
• Build automated data ingestion and transformation pipelines.
• Prepare data using Parquet and Delta formats.
• Sourcing/integrating data from Oracle, Ingres, Azure and SQL Server.
• Leverage cloud data engineering, storage and analytics services.
• Create Power BI visualisations to meaningfully represent data.
• Work with ADF, Databricks, Data Lake/Lakehouse, Azure ML and ArcGIS Enterprise where possible.

Skills

ETL/ELT

Spark

Data pipelines

Parquet/Delta

Oracle data

Ingres data

Azure data

SQL Server data

Cloud data engineering

Power BI

ADF

Databricks

Data Lake/Lakehouse

Azure Machine Learning

ArcGIS Enterprise

Tools

Azure Data Factory

Databricks

Azure Synapse

Job description

One of our Federal Government Clients is seeking to engage a Lead Data Engineer. This is a long-term contractual position.

Please check below all the job details:

Contract Duration: 03 Years (12 months initially + 24 months extension)

Work Location: Canberra (03 Days office/02 Days home per week)

Eligibility: You must need to have Baseline Security Clearance or above.

Position Description:

• Strong hands-on ETL/ELT development for complex/large-scale data ingestion and transformation.
• Spark / distributed computing and optimisation of data for cloud query performance.
• Building automated data ingestion and transformation pipelines/frameworks.
• Data preparation using Parquet and/or Delta formats.
• Experience sourcing/integrating data from Oracle, Ingres, Azure and SQL Server.
• Experience across cloud data engineering, storage and analytics services.
• Power BI – ability to analyse complex datasets and develop meaningful visualisations.
• Highly desirable: Strong hands-on ADF + Databricks, Data Lake/Lakehouse architecture, Azure Machine Learning and/or ArcGIS Enterprise.

Referral incentive program: As always, we have the candidate referral incentive program through which you will get $1000 for each successful referral after the successful selection and joining of the referred candidates.