Senior Data Engineer - Cloud Data Platform Lead

Manulife/John Hancock; Makati; Hybrid; PHP 1,200,000 - 2,100,000; Full time

Job summary

The Asia Data Office at Manulife/John Hancock is seeking an experienced Senior Data Engineer to support our regional partners and data operations. You will drive data quality, mature agile practices, and collaborate across teams to implement ETL/ELT flows using Spark, Python, and Azure services.

You will work on the Enterprise Data Lake (EDL) platform, monitor performance, write documentation, and participate in incident response and change management as part of a data-driven culture.

Qualifications

• 8 years of relevant experience in Data Engineer, Platform Engineer, DevOps Engineer and Software Engineer.
• Proficient understanding of distributed computing principles.
• Good Knowledge of Hadoop cluster, Azure, with all included services.
• Ability to solve ongoing issues with operating the cluster.
• Experience with Azure ADLS, Data Factory, Databricks, Event Hubs.
• Experience with data engineering projects i.e. integration of data from multiple data sources to a target system.
• Good knowledge of data querying tools – Azure Synapse Analytics, Databricks SQL Warehouse.
• Knowledge in data streaming (e.g. Event Hubs, Spark).
• Knowledge in Python and Spark.
• Knowledge in any Change Data Capture (CDC) solutions (e.g. Debezium).
• Experience in working on Agile projects is an advantage.
• Experience on Azure Cloud and/or other Cloud platforms.
• Database development and management (e.g. Azure SQL, Cosmos).
• Good knowledge of insurance Data Domains (Policy, Claims, Underwriting, Customer, Finance & Distribution).
• Strong understanding of Data Governance, Regulations and Compliance.

Responsibilities

• Selecting and integrating any Big Data tools and frameworks required providing requested capabilities.
• Implement ETL/ELT flows via Big Data Solutions i.e. Spark, Python, Azure Data Factory, Azure Synapse, Azure Databricks, etc.
• Monitor the performance and advise necessary infrastructure changes or performance tuning in codes.
• Support business users in use of the Enterprise Data Lake (EDL) as part of BAU.
• Perform POC on new integration patterns and solutions.
• Maintain and monitor platform stability and performance of EDL.
• Attends to Incidents and change requests.
• Write and maintain technical documentation.
• Perform unit tests and system integration tests.
• Executes updates, patches, and other activities required to maintain and enhance the operations of the EDL.
• Supports the Agile delivery squads when required.

Skills

Distributed computing

Hadoop

Azure

Azure Data Factory

Databricks

Python

Spark

CDC (Debezium)

Agile methodologies

Azure Synapse

Event Hubs

Data integration

Azure ADLS

Tools

Azure Data Factory

Azure Synapse

Databricks

Azure SQL

Cosmos DB

Event Hubs

Job description

The Asia Data Office at Manulife/John Hancock is seeking an experienced Senior Data Engineer to support our regional partners and data operations. You will drive data quality, mature agile practices, and collaborate across teams to implement ETL/ELT flows using Spark, Python, and Azure services.

You will work on the Enterprise Data Lake (EDL) platform, monitor performance, write documentation, and participate in incident response and change management as part of a data-driven culture.