Lead Data Engineer

The Great Eastern Life Assurance Company Limited; Singapore; On-site; SGD 140,000 - 240,000; Full time

Job summary

The Great Eastern Life Assurance Company Limited is seeking an experienced Lead Data Engineer to design, build, and enhance enterprise data ingestion, transformation, and quality capabilities. You will develop reliable, scalable data pipelines supporting analytics, reporting, regulatory, and AI use cases across the organization.

Lead the Data Foundation team to strategize data platform implementations (on premise and cloud) and collaborate with IT, governance, and business users to deliver

Qualifications

• Bachelor’s degree in computer engineering, Computer Science, Mathematics, Software Engineering, or equivalent.
• Minimum 10 years of experience in data engineering with data-driven solutions.
• Strong SQL, Spark/PySpark, Hadoop experience; ability to manage large data volumes.
• Experience with data models, ETL, and data platform migrations is preferred.
• Prior experience with banking/insurance data is a plus.

Responsibilities

• Design and develop end-to-end data engineering solutions including ingestion, transformation, and provisioning for analytics and reporting.
• Adopt best practices to build reliable data marts in Hadoop ecosystem for planning and analytics.
• Build and maintain reusable enterprise data assets and canonical data models.
• Collaborate with stakeholders, data scientists and analysts to align data requirements and validate quality.
• Ensure data pipelines are up-to-date with data accuracy and governance policies.
• Support production deployment of data pipelines and data governance adherence.

Skills

SQL

PySpark

Hadoop

GIT

Data governance

Data quality

Education

Bachelor’s degree in Computer Science / Engineering / Math

Tools

SQL

Hive

Cloudera

Hadoop

Spark

Job description

In Great Eastern, the Data Engineering and Business Intelligence team enable data practitioners with Data and Analytics assets. We build datamarts and visual dashboards to realize the full value of our data. We are seeking an experienced Data Engineer to join the Data Foundation team to design, build, and enhance enterprise data ingestion, transformation, and data quality capabilities. The role will focus on developing reliable, scalable, and reusable data pipelines that support analytics, reporting, regulatory, and AI use cases across the organization. The right candidate will have strong knowledge of Data Engineering and proven ability to strategize the implementation of data and analytics capabilities on a data platform (on premise and cloud) from conception through release and production. The ideal candidate will be a strong data engineer; be willing to wrangle data, optimize data systems and products, and build them from the ground up. Experience with insurance industry or Cloud technology (AWS / Google) would be an advantage.

The Lead Data Engineer will work across IT, Data Management and Governance, Visual Analytics, Data Scientists and key Business users to deliver business analytics contextual datasets and data products that enable data-driven businesses.

• Design and develop end-to-end data engineering solutions, including data ingestion, transformation, integration, and provision of trusted data assets for analytics, reporting, and business consumption.
• Adopt the best data engineering practices to design and build reliable data marts in the Hadoop ecosystem for planning, reporting, and analytics
• Build and maintain reusable enterprise data assets, canonical data models, and common data services to promote consistency, scalability, and reuse across multiple business domains.
• Work closely with business stakeholders, data scientists and data analysts to communicate data requirements, collect data, and validate quality of data products
• Collaborate with business stakeholders, data scientists and data analysts to align logic of key metrics, and to ensure code logics correctly reflect latest business definitions
• Maintain and optimize data pipelines to ensure all data is up to date with data accuracy and integrity
• Work across various stakeholders to ensure smooth production deployment of data pipeline and adherence to data governance policies.
• Proactively identify and suggest solutions to improve data engineering proces

• Bachelor’s degree in computer engineering, Computer Science, Mathematics, Software Engineering, equivalent fields or proven experience in data engineering
• Stakeholder Management – Conversant in Business terms and ability to resolve and explain data analytics issues with Business users and other concerned stakeholders
• A minimum of 10 years of experience in data engineering field. An analytics practitioner with proven experience in delivering data-driven business solutions and data-driven process augmentation
• The candidate must have demonstrated experience working with varied data infrastructure inclusive of relational databases such as SQL, Hive, Cloudera, Hadoop, Spark
• Hands-on experience with large volumes of data using Strong SQL, Spark/PySpark, Hadoop, or other big data ecosystems is preferred o Process oriented, and able to translate complex problems into logical and repeatable processes and diligently document the proposed technical solution

• Prior experience in creating, managing data models, ETL and data platforms migration is preferred

• P

  rior experience in managing codes using GIT Version control ,

  Understanding of banking, insurance and financial services is preferred

• Minimum 10 years of working experience in SQL / Hive QL / Spark
• Minimum 10 years of programming experience in preferred languages Python, PySpark
• Minimum 10 years of experience in data engineering work on Big Data Technology (Hortonworks / Cloudera)
• Knowledge of data quality principles and experience implementing data quality controls, validation frameworks, reconciliation processes, and monitoring mechanisms within data pipelines.
• Understanding of metadata management, data lineage, and data governance concepts.
• Experience working with enterprise data models, reusable data assets, and data integration frameworks is preferred.
• Able to work with specialists in different disciplines to formulate data solutions
• Able to work collaboratively as part of a team.
• High level of integrity, takes accountability of work and good attitude over teamwork.
• Takes initiative to improve current state of things and adaptable to embrace new changes.