Data Lake Platform Engineer | Hadoop & Spark

UNISYNC SYSTEMS PTE. LTD.; Singapore; Presencial; SGD 90.000 - 130.000; Jornada completa

Descripción de la vacante

UNISYNC SYSTEMS PTE. LTD. is seeking a seasoned Data Engineer to build and optimize scalable data pipelines on Hadoop platforms, enabling enterprise analytics and data lake consumption across core banking domains.

The role emphasizes design, development, and operational excellence, spanning batch and near real-time ingestion, data quality, governance, and DevOps integration to ensure reliable, high-performance data processing.

Formación

• Design, develop and maintain scalable data pipelines in Enterprise Data Lake platform using CDP, Hadoop, Spark, Hive, Java, Python and SQL.
• Develop high-performance batch and near real-time ingestion frameworks for onboarding enterprise source systems into the Data Lake.
• Create complex transformation logic with Spark, HiveQL, SQL and Java-based processing frameworks.
• Design ingestion patterns for structured, semi-structured and unstructured data.
• Build and maintain batch and near real-time data integration solutions.
• Implement source-to-target mappings and ingestion specifications.

Responsabilidades

• Develop and support data pipelines within the Cloudera Data Platform ecosystem.
• Utilize Hadoop HDFS, Hive, Impala, Spark, Kafka, YARN, and Oozie.
• Conduct deployment, troubleshooting and performance tuning of Hadoop-based applications.
• Collaborate with platform teams to optimize cluster utilization and workload management.
• Provide L3 production support and incident resolution for Data Lake applications.
• Perform data quality checks, governance adherence and issue resolution.

Conocimientos

Java

Python

SQL

Hadoop

Spark

Hive

Kafka

Unix/Linux scripting

Shell scripting

ETL/ELT

Data modelling

Data Warehousing

Data Lake

Herramientas

Git

Jenkins

Azure DevOps

Cloudera CDP

Hadoop

Spark

Descripción del empleo

UNISYNC SYSTEMS PTE. LTD. is seeking a seasoned Data Engineer to build and optimize scalable data pipelines on Hadoop platforms, enabling enterprise analytics and data lake consumption across core banking domains.

The role emphasizes design, development, and operational excellence, spanning batch and near real-time ingestion, data quality, governance, and DevOps integration to ensure reliable, high-performance data processing.