Cloud-Native Data Engineer | Spark, Iceberg & Streaming

OVERSEA-CHINESE BANKING CORPORATION LIMITED; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

OCBC invites an experienced Senior Data Engineer to design cloud-native data pipelines across major platforms. You will build scalable lakehouse solutions with Apache Iceberg and modern governance through OpenMetadata/DataHub, while stream processing in Spark.

You will collaborate with analysts and data scientists, migrate SQL workloads to cloud architectures, and implement automated data quality checks with Python and SQL.

Qualifications

• 3+ years of data engineering experience in cloud-native environments.
• Hands-on with Spark, PySpark, and Iceberg.
• Experience with Open Data Catalogue/Governance (OpenMetadata, DataHub).
• Proficient in Python and SQL; Unix/Linux scripting.
• Bachelor's degree in CS/IT or related field; banking experience preferred.
• Strong collaboration and communication skills.

Responsibilities

• Design and implement scalable data pipelines on cloud platforms.
• Build lakehouse solutions using Iceberg and metadata/catalogue frameworks.
• Develop streaming pipelines and real-time processing.
• Migrate SQL workloads to cloud-native architectures.
• Implement data quality checks and monitoring.
• Collaborate with analysts and data scientists to deliver solutions.
• Participate in code reviews and improve data engineering practices.

Skills

Spark

PySpark

Python

SQL

Unix/Linux scripting

Cloud architecture

Education

Bachelor's degree in Computer Science/Information Technology or related field

Tools

Apache Iceberg

OpenMetadata

DataHub

Open Data Catalogue

Git

Job description

OCBC invites an experienced Senior Data Engineer to design cloud-native data pipelines across major platforms. You will build scalable lakehouse solutions with Apache Iceberg and modern governance through OpenMetadata/DataHub, while stream processing in Spark.

You will collaborate with analysts and data scientists, migrate SQL workloads to cloud architectures, and implement automated data quality checks with Python and SQL.