Senior Cloud Data Engineer: Spark, Iceberg & Real-Time

OCBC (Singapore); Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

OCBC seeks a Senior Data Engineer with cloud-native data engineering expertise on major cloud platforms. You will design scalable data pipelines, implement lakehouse solutions with Apache Iceberg, and build real-time streaming using Spark Structured Streaming.

You will modernize SQL workloads, develop end-to-end ETL/ELT workflows with Python and shell scripting, and contribute to data governance using Open Data Catalogue/OpenMetadata/DataHub.

Qualifications

• A degree in Computer Science, Information Technology, or a related field.
• Minimum 3 years of relevant industry experience in software/data engineering.
• Proficient in Java, Python, or C++ with strong problem-solving.
• Experience with Agile development and Git version control.
• Knowledge of cloud-native data engineering and streaming systems is a plus.

Responsibilities

• Cloud-Native Data Engineering: Design and build scalable data pipelines using native cloud services.
• Streaming & Real-Time Processing: Develop real-time streaming pipelines using cloud-native tools.
• Migration & Modernization: Migrate SQL workloads and legacy systems to cloud-native architectures.
• Data Engineering & Automation: Build end-to-end ETL/ELT workflows with Python and shell scripting; write efficient SQL.
• Data Governance & Catalogue: Leverage Open Data Catalogue, OpenMetadata, DataHub for lineage and metadata capture.
• Collaboration & Delivery: Work with analysts and data scientists to deliver robust data solutions.

Skills

Java

Python

C++

SQL

Spark

PySpark

Education

Bachelor's degree in Computer Science or related field

Tools

Apache Iceberg

OpenMetadata

DataHub

Job description

OCBC seeks a Senior Data Engineer with cloud-native data engineering expertise on major cloud platforms. You will design scalable data pipelines, implement lakehouse solutions with Apache Iceberg, and build real-time streaming using Spark Structured Streaming.

You will modernize SQL workloads, develop end-to-end ETL/ELT workflows with Python and shell scripting, and contribute to data governance using Open Data Catalogue/OpenMetadata/DataHub.