Data Engineer

ESS; Sydney; Hybrid; CAD 80,000 - 110,000; Full time

Job summary

A leading Australian Institute is seeking a Data Engineer to join their Data Analytics and AI team. The role involves designing and maintaining data infrastructure using Azure Databricks and transforming raw data into insights. Candidates should have strong skills in SQL, Python, and data modelling, along with experience in DevOps and cloud services. A supportive and innovative work environment is promised with hybrid working options.

Benefits

Hybrid / flexible working arrangements.

Access to salary sacrificing and insurance.

Personal wellbeing support.

Opportunities for professional development.

Qualifications

• 4 years of experience in data engineering or a related discipline.
• Strong coding skills in SQL and Python for data manipulation.
• Hands-on experience building scalable high-performance data architectures.

Responsibilities

• Design, build and maintain scalable data lakehouse solutions.
• Develop and optimize data ingestion pipelines.
• Deliver robust analytics reporting and dashboards.

Skills

SQL

Python

Data Warehousing

Data Lakehouse Modelling

Azure Databricks

DevOps

Power BI

Education

Tertiary qualifications in a relevant field

Tools

Azure DevOps

Apache Hive

AWS

Hadoop

Spark

NoSQL

Kafka

Job description

Introduction

The Australian Institute of Company Directors (AICD) is committed to strengthening society through world-class governance. We aim to be the independent and trusted voice of governance, building the capability of a community of leaders for the benefit of society. Our membership of more than 50000 includes directors and senior leaders from business, government and the not-for-profit sectors.

Description

About the team

TheData Analytics & AI teamworks to harness the power of data analytics and AI to drive strategic decision-making elevate member experiences and enable a culture of innovation and transparency.

About the Role

We are seeking a Data Engineer to join our growing Data Analytics and AI team. This is a fantastic opportunity to leverage your existing expertise as you will be central to building and maintaining the data infrastructure that powers analytics reporting and AI initiatives. This role involves designing implementing and optimising data models and pipelines using Microsoft Azure and Databricks technologies. A key part of your role will be transforming raw data into reliable timely insights that enable the organisation to make strategic data-informed decisions at scale. Collaborate closely with data scientists analysts and business stakeholders in a fast-paced supportive environment focused on innovation and continuous improvement.

Responsibilities

• Design build and maintain scalable data lakehouse solutions using Azure Databricks and cloud-native tools
• Develop and optimize data ingestion pipelines (batch and real-time) leveraging ETL / ELT best practices
• Perform data cleaning preparation and exploration to ensure data quality and relevance
• Deliver robust analytics reporting and dashboards to support business insights and decision‑making
• Apply dimensional modelling principles (Kimball or Inmon) to design effective data structures
• Collaborate on AI and analytics initiatives ensuring data pipelines meet performance and governance standards
• Implement and uphold data governance security and data quality frameworks
• Drive process improvements and documentation promoting reusable code automation and DevOps practices (including CI / CD)
• Engage with stakeholders to understand and translate business requirements into data solutions

Qualifications

You will have strong technical aptitude and previous relevant experience and you will champion the AICD values of: Accountability, Respect, Excellence and Collaboration. In addition you will also have the following:

• Tertiary qualifications in a relevant field with 4 years of experience in data engineering or a related discipline
• Proficiency in Microsoft Azure cloud services especially Azure Databricks and Azure DevOps
• Strong coding skills in SQL and Python for data manipulation and pipeline development
• Deep understanding of data warehousing and lakehouse modelling principles (Kimball or Inmon)
• Hands‑on experience building scalable high‑performance data architectures supporting batch and streaming data ingestion
• Experience with DevOps processes CI / CD pipelines and infrastructure as code
• Proven ability to perform data cleaning analysis and visualization using tools such as Power BI
• Strong organisation and prioritising skills
• Excellent communication and collaboration skills with a proven track record of working across technical and business teams
• Ability to take ownership of tasks or issues and manage through to resolution

Your career at the AICD

Our Values

Accountability, Respect, Excellence and Collaboration

Benefits

• Hybrid / flexible working arrangements.
• Access to salary sacrificing and salary continuance (income protection) insurance.
• Personal wellbeing support through AICDs robust employee assistance program for you and your immediate family.
• A solutions focused and collaborative organisation that celebrates a One Team culture.
• The AICD is committed to developing its people through a capability framework and ongoing professional development plan. We support your career through access to our education platform with a wide range of training courses and webinars professional coaching and external education support.
• An opportunity to contribute to an organisation that is widely respected and considered the gold standard of governance policy and leadership.
• An opportunity to attend highly sought-after events with influential speakers.

We welcome applications from people of all ages, genders, sexuality, cultural and linguistic backgrounds, Aboriginal and Torres Strait Islander Peoples and people with a disability.

We provide reasonable adjustments including alternate formats to the recruitment process for individuals with disability. If you require an adjustment to be made during the recruitment process please email .

For further information on what its like to work with us please visit our Careers and LinkedIn Life pages :

Our Careers page : LinkedIn page : To Apply

Please submit a resume and cover letter detailing why you are the best person for the position by clicking Apply. Please note applications close on 2 November 2025. We will be in touch with shortlisted candidates shortly after this date. If you have any queries regarding this particular opportunity please contact Lola Perez Lira Talent Acquisition Consultant via email on

Key Skills

Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

Employment Type : Full-Time

Experience : years

Vacancy : 1