Lead Data Engineer | Federal Government

Aurec; Canberra; Vor Ort; AUD 150.000 - 190.000; Vollzeit

Zusammenfassung

Aurec seeks a Lead Data Engineer to join a Federal Government analytics team in Canberra. You will design and maintain cloud-based data ingestion, transformation, and analytics platforms using Azure and Databricks, delivering high-impact operational insights.

The role emphasizes production-grade ETL/ELT pipelines, Parquet/Delta formats, Power BI visualisations, and performance tuning across Spark and Azure SQL, within an Agile environment and NV1 security clearance.

Qualifikationen

• 5+ years of Data Engineer experience with cloud ETL/ELT in Azure and Spark.
• Experience transforming data into Parquet and Delta formats for analytics and AI.
• Proficiency in creating Power BI visualisations for operational analytics.
• Strong communication and collaboration within an Agile delivery team.
• NV1 security clearance or higher is required for federal work.

Aufgaben

• Design, build, and maintain automated ingestion and transformation patterns using modern cloud technologies (Azure & Databricks).
• Transform complex datasets into Parquet and Delta formats tailored for advanced analytics and AI use cases.
• Develop ETL/ELT data pipelines and create Power BI visualisations to support operational analytics and decision-making processes.
• Optimize query performance across distributed computing engines (Spark), Azure SQL, and cloud storage solutions.
• Collaborate within an Agile delivery team while providing quality reviews and constructive feedback to uplift delivery standards.

Kenntnisse

Azure Data Factory

Databricks

SQL Server

Data Lake Storage

Azure ML

DevOps

Visual Studio

Power BI

Parquet

Delta format

Spark

Tools

Spark

Jobbeschreibung

Lead Data Engineer | Federal Government

About the Opportunity

• Join a dynamic, multi-disciplinary section driving critical enterprise analytics and modern data platforms within a secure Federal Government environment.

• Deliver high-impact operational analytics by designing and maintaining cutting-edge ingestion patterns, transformation solutions, and data visualisations using Azure and Databricks technologies.

What's on Offer

• Contract/Engagement: Initial 12-month contract + 1x 12-month extension option.

• Location & Flexibility: Canberra, ACT (Only)

Role & Responsibilities

• Design, build, and maintain automated ingestion and transformation patterns using modern cloud technologies (Azure & Databricks).

• Transform complex datasets into Parquet and Delta formats tailored for advanced analytics and AI use cases.

• Develop ETL/ELT data pipelines and create Power BI visualisations to support operational analytics and decision-making processes.

• Optimize query performance across distributed computing engines (Spark), Azure SQL, and cloud storage solutions.

• Collaborate within an Agile delivery team while providing quality reviews and constructive feedback to uplift delivery standards.

Skills & Experience

To be successful in this position, you will ideally bring:

• Technical Toolkit: Strong expertise in Azure hosted services (Data Factory, Databricks, SQL Server, Data Lake Storage, Azure ML), DevOps, Visual Studio, Power BI, and data formats like Parquet and Delta.

• Experience: 5+ years of experience as a Data Engineer, with proven expertise developing complex ETL/ELT processes in cloud environments and distributed computing engines (e.g., Spark).

• Core Capabilities: High productivity, ownership, and exceptional written and oral communication skills to engage and negotiate effectively with multi-disciplinary team members.

Eligibility & Clearance Requirements

• Security Clearance: Due to the nature of this environment, candidates must hold a valid Negative Vetting Level 1 (NV1) security clearance.