Data Lake Engineer: Scalable Pipelines & Real-Time Data

Unison Group; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

Unison Group seeks a data engineer to design and deliver scalable data pipelines within a robust Data Lake platform on Hadoop/Spark ecosystems. You will champion ingestion, transformation, and governance across structured, semi-structured, and streaming data for enterprise analytics.

You will work with Java, Python, SQL, Spark, and CI/CD tools to ensure reliable, high-performance data processing and deployment automation. Strong collaboration with platform teams is expected.

Qualifications

• Strong experience designing and building scalable data pipelines in a data lake/enterprise data platform.
• Proficient in Java, Python, SQL, and Spark for batch and streaming data processing.
• Hands-on with Hadoop ecosystem components (HDFS, Hive, Impala) and data governance basics.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Cloudera CDP/Hadoop stack.
• Develop high-performance batch and near real-time ingestion frameworks.
• Implement data ingestion mappings and transformations with Spark and HiveQL.

Skills

Java

Python

SQL

Spark

Hadoop

Unix/Linux

Shell scripting

Tools

Hadoop

Hive

Spark

Kafka

Oozie

Git

Jenkins

Azure DevOps

Job description

Unison Group seeks a data engineer to design and deliver scalable data pipelines within a robust Data Lake platform on Hadoop/Spark ecosystems. You will champion ingestion, transformation, and governance across structured, semi-structured, and streaming data for enterprise analytics.

You will work with Java, Python, SQL, Spark, and CI/CD tools to ensure reliable, high-performance data processing and deployment automation. Strong collaboration with platform teams is expected.