Senior Data Engineer on Azure Databricks & AI

AIA Shared Services; Kuala Lumpur; Hybrid; MYR 180,000 - 320,000; Full time

Job summary

AIA Shared Services in Kuala Lumpur is seeking a hands-on Senior Data Engineer to design, implement, and integrate advanced data pipelines and AI solutions across the enterprise. You will act as technical lead for data engineering initiatives, manage end-to-end delivery, govern security and governance, and collaborate with application teams for scalable platforms.

Required: 8–10 years in data engineering, PySpark, SQL, Delta Lake, Databricks, Azure Databricks, and cloud skills; insurance domain

Qualifications

• 8–10 years in data engineering design and development.
• Experience planning end-to-end delivery across projects.
• Strong engagement with business users and stakeholders.
• Proven Azure Databricks data engineering experience.

Responsibilities

• Design, build, and optimize scalable data pipelines and architectures.
• Lead integration with Enterprise data platforms for digital applications.
• Define BAU support model post-implementation and drive continuous improvement.
• Manage vendor relationships and governance for security and performance.
• Stay current with data engineering tech and champion best practices.

Skills

PySpark

SQL

Delta Lake

Databricks Platform

Lakehouse Architecture

Education

Bachelor of Science in Computer Science or related

Tools

Azure Databricks

ADLS Gen2

Azure Data Factory

Azure DevOps

Structured Streaming (Kafka/Event Hub)

Job description

AIA Shared Services in Kuala Lumpur is seeking a hands-on Senior Data Engineer to design, implement, and integrate advanced data pipelines and AI solutions across the enterprise. You will act as technical lead for data engineering initiatives, manage end-to-end delivery, govern security and governance, and collaborate with application teams for scalable platforms.

Required: 8–10 years in data engineering, PySpark, SQL, Delta Lake, Databricks, Azure Databricks, and cloud skills; insurance domain