Senior Data Engineering Lead – Azure, Databricks, Spark

Leadingnation; Hong Kong; On-site; HKD 900,000 - 1,300,000; Full time

Job summary

Leadingnation is seeking an experienced data engineering leader to own delivery for Health IT initiatives in Hong Kong. You will translate business needs into scalable data platforms and clear roadmaps, partnering with Health leaders and IT stakeholders.

You will design and optimize batch and real-time pipelines on Azure and Databricks, drive governance, estimates, and code reviews, ensuring secure and cost-efficient solutions for enterprise-scale workloads.

Qualifications

• 8+ years IT experience with 5+ years in data engineering delivery leadership or technical lead roles.
• Experience with Agile methodology.
• Hands-on Azure Cloud Services, Oracle/SQL Server/PostgreSQL, Azure Databricks, Spark, PySpark, Spark SQL, ETL/ELT design, AliCloud DataWorks, MaxCompute, OSS, Hologres.
• Proficient in written and spoken English; local candidates only (visa sponsorship not available).

Responsibilities

• Partner with Health leaders, product owners and IT to capture requirements, define scope and deliver timelines.
• Collaborate with Health Regional teams.
• Break down initiatives into epics and user stories with estimates.
• Design, build and optimize batch and real-time data pipelines, data lakehouse architectures and warehousing solutions.
• Provide technical leadership for enterprise platforms on Azure and Databricks, drive best practices and cost optimization.
• Establish data governance and quality standards; conduct code reviews and troubleshoot pipelines.

Skills

Data engineering leadership

Azure Databricks

PySpark / Spark SQL

Agile methodologies

Tools

AliCloud DataWorks

MaxCompute

OSS

Hologres

Job description

Leadingnation is seeking an experienced data engineering leader to own delivery for Health IT initiatives in Hong Kong. You will translate business needs into scalable data platforms and clear roadmaps, partnering with Health leaders and IT stakeholders.

You will design and optimize batch and real-time pipelines on Azure and Databricks, drive governance, estimates, and code reviews, ensuring secure and cost-efficient solutions for enterprise-scale workloads.