Lead Data Engineer (Banking)

Hong Kong Unison Limited; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Hong Kong Unison Limited in Singapore is seeking a senior data engineering lead to drive the production delivery of complex data pipelines inside banking environments. You will work with customer, account, transaction and AML data, ensure release governance, and mentor a team of engineers.

Required skills include Teradata SQL, Spark, Hive, Python and SQL tuning, along with Airflow, Git, Kubernetes and CI/CD. This role offers an opportunity to shape enterprise data platforms for major banks.

Qualifications

• Production delivery of data pipelines inside banks.
• Experience with customer, account, transaction, payments and AML data.
• Ability to work within bank release, scheduling and change controls.
• Expertise in Teradata SQL, Spark, Python and R; performance tuning.
• Experience with large-scale data volumes and historical data processing.

Responsibilities

• Framework design, code standards, code reviews and estimation.
• Guide a team of engineers and collaborate with architects and analysts.

Skills

Lead and mentoring

Banking domain expertise

Data pipeline delivery

Stakeholder communication

Tools

Teradata SQL

BTEQ

TPT

FastLoad/MultiLoad

Spark (PySpark/Scala)

Hive

Impala

HBase

Pig

Hue

Apache Iceberg

Trino

Python

R

Kafka

Spark Streaming

Informatica

Talend

Airflow

Control-M

Autosys

Git, CI/CD, GitOps

Kubernetes

Databricks

Delta Lake

Unity Catalog

Workflows

Power BI / Qlik Sense

Job description

Requirements

Experience:

• Lead: 12+ years, including 6+ in banking.
• Senior: 8+ years, including 4+ in banking.

• Production delivery of data pipelines inside banks.
• Customer, account, transaction, payments and AML data.
• Working within bank release, scheduling and change controls.
• Teradata SQL, BTEQ, TPT and FastLoad/MultiLoad; query tuning.
• Spark (PySpark/Scala), Hive, Impala, HBase, Pig and Hue; Apache Iceberg and Trino; Python and R.
• Historical data processing: snapshots, SCD Type 2 and multi-grain backfills with control totals and validation reports.
• ETL/ELT development, CDC, metadata-driven automation frameworks and parameter-driven SQL generation.
• Reconciliation, data quality and SLA monitoring.
• Performance at scale: tables of 1 billion+ rows and multi-year history.

Banking and domain

• Production delivery of data pipelines inside banks.
• Customer, account, transaction, payments and AML data.
• Working within bank release, scheduling and change controls.
• Teradata SQL, BTEQ, TPT and FastLoad/MultiLoad; query tuning.
• Spark (PySpark/Scala), Hive, Impala, HBase, Pig and Hue; Apache Iceberg and Trino; Python and R.
• Historical data processing: snapshots, SCD Type 2 and multi-grain backfills with control totals and validation reports.
• ETL/ELT development, CDC, metadata-driven automation frameworks and parameter-driven SQL generation.
• Reconciliation, data quality and SLA monitoring.
• Performance at scale: tables of 1 billion+ rows and multi-year history.

Integration and platforms

• Kafka, Spark Streaming, Informatica (PowerCenter, IDMC, IDL) and Talend; REST API development.
• Denodo, Snowflake and NoSQL databases.
• Airflow, Control-M or Autosys; Git, CI/CD, GitOps and Kubernetes.

Delivery and communication (Lead)

• Framework design, code standards, code reviews and estimation.
• Guiding a team of engineers and working with architects and analysts.

Good-to-have skills

• Databricks: Delta Lake, Unity Catalog and Workflows.
• Data services on Azure, Google Cloud, Huawei Cloud or Alibaba Cloud.
• Data modelling for Qlik Sense or Power BI.

Certifications (preferred): Teradata Vantage; Cloudera Data Engineer; Databricks Data Engineer Associate or Professional

Unison Consulting established in the year 2012 in Singapore. We provide technology consulting, systems implementation, and integrations service to Asia Pacific customers. We help transform enterprise data in actionable insights. Our consultants have wide range of functional expertise within Banking and Financial services spectrum, working closely with clients to anticipate, illuminate and overcome complex business challenges.

Our objective is to be a one-stop solution provider all enterprise business intelligence and data warehousing needs of organizations.

Our mission is to consistently deliver our services to our clients in a cost-effective way without compromising the quality.

Our current clients include - DBS, OCBC, Standard Chartered Bank and Maybank.