Lead Data Engineer

Indigitise; Canberra; Hybrid; AUD 110,000 - 140,000; Full time

Job summary

Indigitise is seeking a Data Engineer to join our federal client’s Enterprise Analytics and Technology Services. The role focuses on ETL/ELT development, data products, and complex data visualisation across cloud data platforms.

You will work in a hybrid Canberra setup with three in-office days weekly, promoting agile ceremonies and essential team collaboration. You will design, build and test data movement, transformation and visualisation processes to enable operational analytics.

Qualifications

• 5+ years’ experience as a data engineer.
• Experience in ETL/ELT and data products development.
• Azure & Databricks experience.
• Ability to work in an Agile team.
• Strong communication and leadership capabilities.

Responsibilities

• Be responsive, flexible, and work collaboratively as part of an agile team.
• Build strong relationships and negotiate effectively with stakeholders.
• Demonstrate strong written and oral communication skills.
• Create and maintain automated ingest and transformation patterns and frameworks.
• Design data ingest and transformation solutions for current and emerging needs.
• Support project teams to achieve departmental objectives and priorities.
• Assist data engineers in delivery teams and provide constructive feedback.

Skills

ETL/ELT development

Data visualization

Agile teamwork

Leadership

Stakeholder relations

Data analytics interpretation

Education

Data engineering training

Tools

Data Factory

SQL Server Integration Services

Databricks

SQL Server

Data Lake Storage

DevOps

Visual Studio

Oracle

Ingres

Azure

Parquet

Delta

Job description

Canberra, Australia

HybridMost of the section is located at Agriculture House in Canberra CBD. Standard expectation is three (3) working days in the office each week. Flexible arrangements may be considered. In-person attendance at important Agile ceremonies and other key team/section events is required.

Job Description

Our federal client is looking for a Data Engineer to join itsEnterprise Analytics and Technology Services (EATS) section to work across itsdata and analytics platforms. We are seeking candidates with strong experiencein developing Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT)processes, and /or the development of data products and complex datavisualisations.

The role will be responsiblefor design, development and testing activities across several data movement,data transformation, and data visualisation processes within DAFF. The datamovement and transformation processes focus on the preparation of data foruseid decision making processes across the department, utilising modern cloudtechnology (Azure & Databricks) to enable operational analytics use cases.

To be successful in the role,you must have a strong work ethic including taking ownership, providingleadership, having a high level of productivity, and working with amulti-disciplinary team in an Agile development environment. You must havesuitable qualifications or training in data engineering techniques and at least5 years’ experience working as a data engineer.

Requirements

Responsibilities include, but not limited to:

• Be responsive,flexible, and work collaboratively as part of an agile team.
  
• Strongrelationship building, and negotiation skills.
  
• Strong written andoral communication skills.
  
• Creating andmaintaining automated ingest and transformation patterns and frameworks.
  
• Designing,building and maintaining data ingest and transformation solutions to meetcurrent and emerging needs.
  
• Assisting projectteams achieve objectives that align with departmental, divisional, andprogram priorities.
  
• Supporting dataengineers in delivery teams, through regular quality reviews andconstructive feedback on utilising data assets to produce quality dataproducts.
  

The successful candidate willrequire experience with the following techniques and technologies:

• DataIntegration - Data Factory, SQL Server Integration Services and/orDatabricks
  
• DataStore - SQL Server and/or Data Lake Storage
  
• Development tools– DevOps, Visual Studio
  
• Data technologysolutions – sourcing (Oracle, Ingres, Azure, SQL Server), automatedingestion
  
• Data Preparation
  
• Transformationof data into formats tailored for advanced analytics and AI use cases –Parquet and/or Delta
  
• Analyseand interpret complex data sets, and to identify trends and patterns.
  
• Designprinciples to create appropriate visualisations for target audience.