Senior Data Engineer — Azure Big Data & EDL

Manulife; Philippines; Hybrid; PHP 1,000,000 - 2,100,000; Full time

Job summary

Manulife is seeking an experienced Senior Data Engineer to support regional partners in building high-quality data solutions. The role involves ETL/ELT with Spark, Python, Azure Data Factory, Synapse and Databricks within the Enterprise Data Lake environment.

You will monitor performance, propose infrastructure improvements, and collaborate with Agile teams to deliver robust data platforms and documentation. Hybrid work arrangement in the Asia Data Office region is offered.

Qualifications

• 8 years of relevant experience in Dat Engineer, Platform Engineer, DevOps Engineer and Software Engineer
• Proficient understanding of distributed computing principles
• Good knowledge of Hadoop cluster, Azure, with all included services
• Ability to solve ongoing issues with operating the cluster
• Experience with Azure ADLS, Data Factory, Databricks, Event Hubs
• Experience with data engineering projects i.e. integration of data from multiple data sources to a target system
• Good knowledge of data querying tools – Azure Synapse Analytics, Databricks SQL Warehouse
• Knowledge in data streaming (Event Hubs, Spark)
• Knowledge in Python and Spark
• Knowledge in Change Data Capture (CDC) solutions (e.g. Debezium)
• Experience in working on Agile projects is an advantage
• Experience on Azure Cloud and/or other Cloud platforms
• Database development and management (e.g. Azure SQL, Cosmos)
• Good knowledge of insurance Data Domains (Policy, Claims, Underwriting, Customer, Finance & Distribution)
• Strong understanding of Data Governance, Regulations and Compliance

Responsibilities

• Select Big Data tools and frameworks required for capabilities
• Implement ETL/ELT flows via Spark, Python, Azure Data Factory, Synapse, Databricks
• Monitor performance and advise infrastructure changes or tuning
• Support business users in Enterprise Data Lake usage
• Perform POC on new integration patterns and solutions
• Maintain platform stability and performance of EDL
• Attend incidents and change requests
• Write and maintain technical documentation
• Perform unit tests and system integration tests
• Execute updates, patches to maintain operations of the EDL
• Support Agile delivery squads when required

Skills

Distributed computing

Agile projects

Insurance data domains

Tools

Hadoop cluster

Azure Data Factory

Azure Synapse

Azure Databricks

Event Hubs

Azure ADLS

Python

Spark

CDC (Debezium)

Cosmos

Azure SQL

Job description

Manulife is seeking an experienced Senior Data Engineer to support regional partners in building high-quality data solutions. The role involves ETL/ELT with Spark, Python, Azure Data Factory, Synapse and Databricks within the Enterprise Data Lake environment.

You will monitor performance, propose infrastructure improvements, and collaborate with Agile teams to deliver robust data platforms and documentation. Hybrid work arrangement in the Asia Data Office region is offered.