Hadoop Data Engineer for Real-Time Analytics

Unison Group; Kuala Lumpur; On-site; MYR 120,000 - 240,000; Full time

Job summary

Unison Group in Kuala Lumpur is seeking a Big Data Hadoop Developer to design, develop, and maintain large-scale data processing solutions. The role focuses on hands-on work with the Hadoop ecosystem and strong integration with MariaDB or Oracle DB for analytics.

You will design, implement, and optimize Hadoop-based solutions, work with data ingestion tools like Sqoop, Kafka, and Nifi, and collaborate with BI and data science teams to ensure secure, governed data delivery.

Qualifications

• Strong hands-on experience with Hadoop ecosystem for large-scale data processing.
• Experience with data ingestion frameworks to pull data from relational databases.
• Proficiency in Hive/Spark/MapReduce for transformations and analytics.
• Ability to optimize Hadoop jobs and HDFS usage for performance and cost efficiency.

Responsibilities

• Design, develop, and optimize Hadoop-based data solutions for batch and real-time processing.
• Integrate data from MariaDB/Oracle into Hadoop using Sqoop, Nifi, and Kafka.
• Implement Hive, Spark, and MapReduce jobs for data transformation and analytics.
• Tune Hive queries and Spark jobs; optimize data layout and storage.
• Create and maintain ETL pipelines for structured and unstructured data.
• Troubleshoot Hadoop jobs and database connectivity issues.
• Collaborate with BI, analytics, and data science teams for data provisioning.
• Ensure data security, governance, and compliance in all solutions.

Skills

Hadoop

Hive

Spark

Java/Python/Scala

SQL

Tools

Sqoop

Kafka

Nifi

Ambari

Cloudera Manager

Git

Jenkins

Linux Shell

Job description

Unison Group in Kuala Lumpur is seeking a Big Data Hadoop Developer to design, develop, and maintain large-scale data processing solutions. The role focuses on hands-on work with the Hadoop ecosystem and strong integration with MariaDB or Oracle DB for analytics.

You will design, implement, and optimize Hadoop-based solutions, work with data ingestion tools like Sqoop, Kafka, and Nifi, and collaborate with BI and data science teams to ensure secure, governed data delivery.