Lead Data Engineering – Azure Databricks

AIA Hong Kong and Macau; Australia; On-site; AUD 180,000 - 240,000; Full time

Benefits offered by this job

Flexible working arrangements

Well-being initiatives

Career development opportunities

Job summary

AIA Australia is seeking a Technical Lead – Data Engineering (Azure Databricks) to guide the data engineering team and shape their Azure-based analytics platform. The role is anchored in Sydney or Melbourne and reports to the Head of Data Platforms and Analytics.

You will design scalable data pipelines, lead architecture efforts, and mentor engineers to deliver high-impact data solutions within the insurance/financial services domain.

Qualifications

• Extensive experience in data design and engineering, preferably within insurance, banking, or financial services.
• Hands-on experience developing data engineering and architecture frameworks using Databricks, data modelling, and ETL automation.
• Proficiency in Python and SQL.
• Experience with Azure cloud services (ADF, ADLS, Azure SQL).
• Strong problem-solving skills and the ability to analyse complex technical issues.
• Excellent communication skills, able to convey technical concepts to both technical and non-technical audiences.
• Strong leadership skills, including mentoring and developing high-performing teams.
• Databricks certification is highly regarded.
• Knowledge or experience in the insurance industry is highly regarded.

Responsibilities

• Define detailed technical solutions and manage delivery of the data engineering team (onshore and offshore) based on business requirements and high-level architecture.
• Design, build, and maintain scalable, efficient data pipelines and frameworks using Azure Databricks for ingestion, curation, and transformation of data.
• Lead data modelling, ETL automation, DevOps, data quality, security, and performance optimisation using best practices on Azure.
• Collaborate with key stakeholders across business, analytics, IT, and vendors to align engineering efforts with business priorities.
• Mentor and coach a team of data engineers and analysts, promoting best engineering practices and upskilling in the Azure Databricks toolset.

Skills

Databricks

Python

SQL

Azure Cloud

Data modelling

ETL automation

Leadership

Mentoring

Communication

DevOps

Tools

ADF

ADLS

Azure SQL

Job description

AIA Australia is seeking a Technical Lead – Data Engineering (Azure Databricks) to guide the data engineering team and shape their Azure-based analytics platform. The role is anchored in Sydney or Melbourne and reports to the Head of Data Platforms and Analytics.

You will design scalable data pipelines, lead architecture efforts, and mentor engineers to deliver high-impact data solutions within the insurance/financial services domain.