Lead Data Engineer | NV1 Cleared, Azure/Databricks

Aurec; Canberra; Presencial; AUD 150 000 - 190 000; Tempo integral

Resumo da oferta

Aurec seeks a Lead Data Engineer to join a Federal Government analytics team in Canberra. You will design and maintain cloud-based data ingestion, transformation, and analytics platforms using Azure and Databricks, delivering high-impact operational insights.

The role emphasizes production-grade ETL/ELT pipelines, Parquet/Delta formats, Power BI visualisations, and performance tuning across Spark and Azure SQL, within an Agile environment and NV1 security clearance.

Qualificações

• 5+ years of Data Engineer experience with cloud ETL/ELT in Azure and Spark.
• Experience transforming data into Parquet and Delta formats for analytics and AI.
• Proficiency in creating Power BI visualisations for operational analytics.
• Strong communication and collaboration within an Agile delivery team.
• NV1 security clearance or higher is required for federal work.

Responsabilidades

• Design, build, and maintain automated ingestion and transformation patterns using modern cloud technologies (Azure & Databricks).
• Transform complex datasets into Parquet and Delta formats tailored for advanced analytics and AI use cases.
• Develop ETL/ELT data pipelines and create Power BI visualisations to support operational analytics and decision-making processes.
• Optimize query performance across distributed computing engines (Spark), Azure SQL, and cloud storage solutions.
• Collaborate within an Agile delivery team while providing quality reviews and constructive feedback to uplift delivery standards.

Conhecimentos

Azure Data Factory

Databricks

SQL Server

Data Lake Storage

Azure ML

DevOps

Visual Studio

Power BI

Parquet

Delta format

Spark

Ferramentas

Spark

Descrição da oferta de emprego

Aurec seeks a Lead Data Engineer to join a Federal Government analytics team in Canberra. You will design and maintain cloud-based data ingestion, transformation, and analytics platforms using Azure and Databricks, delivering high-impact operational insights.

The role emphasizes production-grade ETL/ELT pipelines, Parquet/Delta formats, Power BI visualisations, and performance tuning across Spark and Azure SQL, within an Agile environment and NV1 security clearance.