Data Engineering Tech Lead — Azure Databricks

AIA; City of Melbourne; On-site; AUD 150,000 - 190,000; Full time

Job summary

AIA Australia is embarking on a major technology, data and business transformation. We are building a modern platform on Azure Databricks and seek a Technical Lead - Data Engineering to guide the migration of legacy data warehouses to modern data products.

You will lead onshore and offshore data engineers, design scalable data solutions, and mentor a high-performing team, reporting to the Head of Data Platforms and Analytics. Melbourne or Sydney-based with hybrid options.

Qualifications

• Extensive previous data design related experience including hands-on Data engineering and detailed design, preferably in insurance, banking or financial services industry.
• Demonstrated hands-on experience with developing and implementing data engineering and architecture frameworks and processes using Databricks, data modelling, and automation of ETL processes.
• Demonstrated proficiency with programming languages, such as Python and SQL.
• Experience with Azure cloud platform and their data services like ADF, ADLS, SQL.
• Strong problem-solving skills with the ability to analyze complex technical issues and propose effective solutions.
• Excellent communication skills with the ability to convey technical concepts to both technical and non-technical stakeholders.
• Strong leadership skills, including the ability to mentor, coach, and develop a high-performing data engineering team.
• Certification in Databricks will be highly regarded.
• Knowledge of/previous engineering experience within the Insurance industry would be highly regarded.

Responsibilities

• Define a detailed technical solution and manage technical delivery of the data engineering team (onshore and offshore) based on detailed Business Requirements and aligned to the high level solution architecture data.
• Design, Build and maintain scalable and efficient detailed design and frameworks using Azure Databricks for ingestion, curation and transformation of data.
• Responsible for data modelling, ETL automation, DevOps, data quality, data security and performance optimization leveraging best practices of Azure cloud platform.
• Collaborate with key stakeholders across Business, Analytics, IT teams (local and offshore) and vendors, ensuring alignment of data engineering efforts with business priorities.
• Mentor and support a team of data engineers and analysts, in development of best engineering practice and upskilling engineers in Azure databricks toolset.

Skills

Python

SQL

Data modelling

ETL design

Leadership

Mentoring

Communication

Problem solving

Tools

Databricks

Azure Data Factory

Azure Data Lake Storage

Azure

Job description

AIA Australia is embarking on a major technology, data and business transformation. We are building a modern platform on Azure Databricks and seek a Technical Lead - Data Engineering to guide the migration of legacy data warehouses to modern data products.

You will lead onshore and offshore data engineers, design scalable data solutions, and mentor a high-performing team, reporting to the Head of Data Platforms and Analytics. Melbourne or Sydney-based with hybrid options.