Big Data Hadoop Engineer - Spark, Hive & Analytics

Unison Group; Kuala Lumpur; On-site; MYR 120,000 - 180,000; Full time

Job summary

Unison Group in Kuala Lumpur is seeking a Big Data Hadoop Developer to design, develop, and maintain large-scale data processing solutions using the Hadoop ecosystem, with emphasis on MariaDB/Oracle analytics.

You will implement Hive, Spark, and MapReduce jobs, optimize queries, and build ETL pipelines, collaborating with BI and data science teams while ensuring data security and governance.

Qualifications

• Experience with Hadoop ecosystem for large-scale data processing
• Hands-on experience with relational databases (MariaDB/Oracle) for analytics
• Proficiency in Java/Scala/Python for Spark/MapReduce development

Responsibilities

• Design, develop, and optimize Hadoop-based big data solutions for batch and real-time data processing.
• Work with data ingestion frameworks to integrate data from MariaDB/Oracle DB into Hadoop (Sqoop, Apache Nifi, Kafka).
• Implement Hive, Spark, and MapReduce jobs for data transformation and analytics.
• Optimize Hive queries, Spark jobs, and HDFS usage for performance and cost efficiency.
• Create and maintain ETL pipelines for structured and unstructured data.
• Troubleshoot and resolve issues in Hadoop jobs and database connectivity.
• Collaborate with BI, analytics, and data science teams for data provisioning.
• Ensure data security, governance, and compliance in all solutions.

Skills

Hadoop Ecosystem

Hive

Spark

MapReduce

Java/Scala/Python

Tools

Ambari

Cloudera Manager

Git

Jenkins

Job description

Unison Group in Kuala Lumpur is seeking a Big Data Hadoop Developer to design, develop, and maintain large-scale data processing solutions using the Hadoop ecosystem, with emphasis on MariaDB/Oracle analytics.

You will implement Hive, Spark, and MapReduce jobs, optimize queries, and build ETL pipelines, collaborating with BI and data science teams while ensuring data security and governance.