VP, Senior Data Engineer – Lakehouse & AI Platforms

United Overseas Bank; Singapore; On-site; SGD 180,000 - 300,000; Full time

Job summary

United Overseas Bank invites a hands-on Senior Data Engineer to design and build enterprise-scale ingestion frameworks, Lakehouse data platforms, governed data products and analytical data models using Hive, Impala, Spark, Kafka, Iceberg, Ranger, Atlas, Nifi, Flink, and more. You will partner with architecture, governance, analytics and business teams to deliver trusted, reusable data assets.

You will create metadata-driven ingestion frameworks, reusable Spark/Python components,

Qualifications

• Master’s degree in computer science or related disciplines.
• At least 15 years of experience in technology delivery, preferably within banking or financial services

Responsibilities

• Design metadata-driven ingestion frameworks supporting batch, CDC, streaming, APIs and files.
• Develop reusable Spark/Python components, orchestration, logging, monitoring and error handling.
• Build Bronze, Silver and Gold layers using Lakehouse principles.
• Design conceptual, logical and physical data models.
• Create business-facing data products with metadata, lineage and SLAs.
• Implement data quality controls and observability.
• Performance tune Spark jobs and distributed processing workloads.
• Support DevOps, CI/CD and Infrastructure as Code practices.
• Strong Programming skills (Java, Python, SQL)
• Build robust data ingestion and transformation frameworks using Java, Spark, Python, and shell scripting for ingesting multi-model data (image, audio, video, unstructured documents) with both batch and real-time.
• Strong hands on experience with Kubernetes, OpenShift, Docker, CI/CD, MLflow and observability tools.
• Ability to design data architectures supporting NLP and AI-driven analytics, including ingestion, curation, and governance of unstructured data within Data Lake, Data warehouse platforms.
• Experience working with ML platforms such as CML, Spark MLlib, and Python ML libraries (scikit-learn, XGBoost), including model deployment.
• Design and develop highly scalable, Real time systems using Hadoop ecosystem components(Iceberg, Spark, Ozone, Trino, Hive, Ranger, Kafka, Flink and Nifi)
• Strong experience in building full agentic application with all the features(e.g mcp/tools/memory/rag)
• Experienced Cloud platforms and Cloud based app deployments with observability.
• Strong exposure to AI-driven risk scoring, fraud investigation and explainable AI use cases in banking.
• Develop full-stack applications and internal engineering tools using Python, shell scripting, and modern web frameworks (e.g., Flask, React).
• Collaborate closely with data scientists to operationalize machine learning models using Cloudera Machine Learning (CML).
• Perform performance tuning and optimization of data applications on Hadoop to ensure optimal resource utilization

Skills

Apache Spark (PySpark)

Java

Python

Advanced SQL

Data Modeling

ETL/ELT & CDC

API integration

Delta Lake / Lakehouse concepts

Data governance / metadata

Education

Master's degree in Computer Science

Tools

Kubernetes

OpenShift

Docker

CI/CD

MLflow

Job description

United Overseas Bank invites a hands-on Senior Data Engineer to design and build enterprise-scale ingestion frameworks, Lakehouse data platforms, governed data products and analytical data models using Hive, Impala, Spark, Kafka, Iceberg, Ranger, Atlas, Nifi, Flink, and more. You will partner with architecture, governance, analytics and business teams to deliver trusted, reusable data assets.

You will create metadata-driven ingestion frameworks, reusable Spark/Python components,