Data Platform Engineer - Hadoop & Spark

RISKDATA CONSULTING PTE. LTD.; Singapore; Presencial; SGD 120.000 - 180.000; Jornada completa

Descripción de la vacante

Riskdata Consulting Pte. Ltd. is seeking a senior Data Engineer to design and deliver scalable data pipelines within the Enterprise Data Lake. You will work across Hadoop-based technologies to ingest, transform, and integrate data from multiple enterprise sources for warehouse and analytics consumption.

The role emphasizes batch and near real-time processing, data governance, and DevOps-driven deployment, with production support responsibilities and a focus on performance optimization.

Formación

• Proven experience building data pipelines in large-scale environments.
• Strong knowledge of Hadoop ecosystem including HDFS, Hive, Spark.
• Experience with data ingestion, ETL/ELT, and data quality.
• Experience with Java or Python and SQL.
• Experience with CI/CD and DevOps tooling.

Responsabilidades

• Design and maintain scalable data pipelines in the Enterprise Data Lake.
• Develop batch and near real-time ingestion for multiple source systems.
• Implement data transformation using Spark, HiveQL, and SQL.
• Ensure data governance, quality, and security across data platforms.
• Support deployment, monitoring, and optimization in production.

Conocimientos

Data pipeline design

Batch processing

Real-time processing

Herramientas

Cloudera CDP

Hadoop

Hive

Spark

Kafka

YARN

Oozie

Unix/Linux

Shell scripting

Java

Python

SQL

Descripción del empleo

Riskdata Consulting Pte. Ltd. is seeking a senior Data Engineer to design and deliver scalable data pipelines within the Enterprise Data Lake. You will work across Hadoop-based technologies to ingest, transform, and integrate data from multiple enterprise sources for warehouse and analytics consumption.

The role emphasizes batch and near real-time processing, data governance, and DevOps-driven deployment, with production support responsibilities and a focus on performance optimization.