Lead Data Engineer - Spark & Databricks

AON SINGAPORE CENTER FOR INNOVATION, STRATEGY AND MANAGEMENT PTE. LTD.; Singapore; On-site; SGD 90,000 - 120,000; Full time

Job summary

A global consultancy firm in Singapore is seeking a Lead Data Engineer to design and implement scalable data architectures. The ideal candidate will have 8+ years of experience, with in-depth knowledge of Apache Spark and Databricks, driving data solutions for innovative projects. This role offers a collaborative environment, focusing on data-driven initiatives and continuous learning opportunities to ensure colleagues can thrive.

Benefits

Comprehensive benefits package

Global Wellbeing Days

Smart Working flexibility

Learning and growth opportunities

Qualifications

• 8+ years of experience working with data integration teams.
• 5+ years in data architecture, database design, and data management.
• 4+ years developing data pipelines within an Apache Spark environment.
• 3+ years of hands-on experience with Databricks.
• Knowledge of cloud platforms and big data technologies.
• Experience with BI and analytics tools.

Responsibilities

• Collaborate with actuaries, data analysts, and engineers to deliver projects.
• Design scalable data architectures and implement best practices.
• Drive automation and infrastructure-as-code practices.
• Perform data analysis and troubleshoot data issues.
• Lead the data engineering team in architectural design.

Skills

Apache Spark

Databricks

Pyspark

Python

SQL

Data modelling

Cloud platforms (AWS or Azure)

ETL/ELT processes

Business Intelligence tools (Power BI, Tableau)

Communication skills

Job description

A global consultancy firm in Singapore is seeking a Lead Data Engineer to design and implement scalable data architectures. The ideal candidate will have 8+ years of experience, with in-depth knowledge of Apache Spark and Databricks, driving data solutions for innovative projects. This role offers a collaborative environment, focusing on data-driven initiatives and continuous learning opportunities to ensure colleagues can thrive.