Lead Data Engineer — Cloud Analytics & ETL

Pinaka Technology Solutions Pty Ltd; Canberra; Hybrid; AUD 120,000 - 160,000; Full time

Job summary

The Department of Agriculture, Fisheries and Forestry (DAFF) invites applications for a Lead Data Engineer to join its Enterprise Analytics and Technology Services (EATS) team in Canberra. You will design, develop and test ETL/ELT processes and data products across cloud platforms, with Azure and Databricks playing a central role.

Applicants must have at least 5 years in data engineering, be able to obtain Negative Vetting Level 1 clearance, and be comfortable working in a hybrid Canberra-based

Qualifications

• 5+ years’ experience in data engineering and ETL/ELT in cloud environments.
• Experience preparing data for analytics in distributed/cloud engines (Spark).
• Strong communication and collaboration within cross-functional teams.
• Eligible for Negative Vetting Level 1 clearance.

Responsibilities

• Be responsive, flexible, and work collaboratively as part of an agile team.
• Strong relationship building and negotiation skills.
• Strong written and oral communication skills.
• Creating and maintaining automated ingest and transformation patterns and frameworks.
• Assisting project teams achieve objectives that align with departmental, divisional, and program priorities.
• Supporting data engineers in delivery teams, through regular quality reviews and constructive feedback on utilising data assets to produce quality data products.
• Azure hosted services including: Data Factory, SQL Server Integration Services and/or Databricks.
• Data Store - SQL Server and/or Data Lake Storage.
• Analytics - Azure Databricks, Azure Machine Learning, ArcGIS Enterprise.
• Development tools – DevOps, Visual Studio.
• Data technology solutions – sourcing (Oracle, Ingres, Azure, SQL Server), automated ingestion.
• Data Preparation – Transformation of data into formats tailored for advanced analytics and AI use cases – Parquet and/or Delta.
• Data Visualisation – Analyse and interpret complex data sets, and to identify trends and patterns; Visualisation tools - Power BI.

Skills

ETL/ELT development

Cloud data movement

Data visualisation

Tools

Azure Data Factory

Databricks

SQL Server

Power BI

Job description

The Department of Agriculture, Fisheries and Forestry (DAFF) invites applications for a Lead Data Engineer to join its Enterprise Analytics and Technology Services (EATS) team in Canberra. You will design, develop and test ETL/ELT processes and data products across cloud platforms, with Azure and Databricks playing a central role.

Applicants must have at least 5 years in data engineering, be able to obtain Negative Vetting Level 1 clearance, and be comfortable working in a hybrid Canberra-based