Data Engineering Lead, Azure Databricks Platform

AIA Hong Kong and Macau; Australia; Hybrid; AUD 180,000 - 240,000; Full time

Benefits offered by this job

Training & development

Internal mobility

Well-being & extra leave

Flexible work arrangements

Inclusive environment

Job summary

AIA Australia is seeking a Technical Lead – Data Engineering (Azure Databricks) to drive delivery within a large technology and data platform program. The role can be based in Sydney or Melbourne and will mentor a team of engineers to build scalable data pipelines.

You will work with Python, SQL, and Azure services (ADF/ADLS/Azure SQL) to model data, automate ETL, and ensure robust data quality and security practices.

Qualifications

• Extensive experience in data design and engineering, preferably within insurance, banking, or financial services.
• Hands-on experience developing data engineering and architecture frameworks using Databricks, data modelling, and ETL automation.
• Proficiency in programming languages and tools such as Python and SQL.
• Experience with Azure cloud services (ADF, ADLS, Azure SQL).
• Strong problem-solving skills and the ability to analyse complex technical issues.
• Excellent communication skills, able to convey technical concepts to both technical and non-technical audiences.
• Strong leadership skills, including mentoring and developing high-performing teams.
• Databricks certification is highly regarded.
• Knowledge or experience in the insurance industry is highly regarded.

Responsibilities

• Define detailed technical solutions and manage delivery of the data engineering team (onshore and offshore) based on business requirements and high-level architecture.
• Design, build, and maintain scalable, efficient data pipelines and frameworks using Azure Databricks for ingestion, curation, and transformation of data.
• Lead data modelling, ETL automation, DevOps, data quality, security, and performance optimisation using best practices on Azure.
• Collaborate with key stakeholders across business, analytics, IT, and vendors to align engineering efforts with business priorities.
• Mentor and coach a team of data engineers and analysts, promoting best engineering practices and upskilling in the Azure Databricks toolset.

Skills

Data engineering

Leadership

Communication

Problem solving

Mentoring

Tools

Azure Databricks

Python

SQL

ADF

ADLS

Azure SQL

Job description

AIA Australia is seeking a Technical Lead – Data Engineering (Azure Databricks) to drive delivery within a large technology and data platform program. The role can be based in Sydney or Melbourne and will mentor a team of engineers to build scalable data pipelines.

You will work with Python, SQL, and Azure services (ADF/ADLS/Azure SQL) to model data, automate ETL, and ensure robust data quality and security practices.