Hadoop Developer - Transform Data Driven

eTeam; Kuala Lumpur; On-site; MYR 90,000 - 120,000; Full time

Job summary

A data solutions company in Kuala Lumpur is seeking a Hadoop Developer to support the development and maintenance of Hadoop solutions. The ideal candidate will have a minimum of 8 years in data-related fields, including 2+ years of Hadoop/ETL experience. Responsibilities include ensuring data quality and optimization of ETL processes, as well as supporting production and maintenance of existing datasets in Hadoop. Strong scripting and communication skills are essential. Join a dynamic team and help drive data initiatives in an enterprise environment.

Qualifications

• Minimum 8 years' experience in data-related work with at least 2+ years as Hadoop/ETL.
• Ability to work independently and with stakeholders.
• Self-motivated, strong interpersonal and communication skills.

Responsibilities

• Support development and maintenance of Hadoop solutions.
• Perform production support and maintenance of existing datasets.
• Be involved in initiatives to bring data into the data lake.

Skills

Project planning

Multi-tasking

Problem-solving

Interpersonal skills

Verbal communication

Written communication

Tools

Linux

SQL

Hadoop

Python

Scala

Spark

Hive

HBase

Sqoop

Oozie

Job description

Hadoop Developer - Transform Data Driven

Qualifications

• Min 8 years' experience in data related work in Warehouse / Data Marts, with atleast 2+ years of experience as Hadoop/ETL experience (data processing and scripting).
• Ability to plan and organize technical work and deliverables.
• Ability to follow guidelines and adhere to the established software development standards and conventions.
• Self-motivated and independent.
• Able to work with minimum supervision and to work well with stakeholders and project staff.
• Ability to prioritize and multi-task across numerous work streams.
• Strong interpersonal skills; ability to work on cross-functional teams.
• Strong verbal and written communication skills.
• Deep knowledge of best practices through relevant experience across data-related disciplines and technologies particularly for enterprise wide data architectures and data warehousing/BI.
• Demonstrated problem-solving skills.
• Ability to learn effectively and meet deadlines.

Technical Skills

• Strong scripting skills in Linux environment and SQL.
• Expertise in Hadoop ecosystems HDFS (Hortonworks).
• Hands-on Experience in Sqoop, Hive, HBase, Spark, Oozie, Python, Scala is a mus
• Develop ETL/ELT jobs based on requirements from source to Data lake platform.
• Ensure that all development standards are being followed.
• Perform code review functions for applications programs developed by team members.
• Provide production and operational support after production deployment.
• Monitor and manage production jobs to verify execution and measure performance to assure ongoing data quality and optimization of the system to manage scalability and performance and identify improvement opportunities for key ETL processes.
• Work effectively with all technical personnel (Development Team, business analysts, security, risk and compliance, data center, project managers, data architects and testers), and clearly translate business priorities and objectives into technical solutions.

Good to have skills

• Develop ETL/ELT jobs based on requirements from source to Data lake platform.
• Ensure that all development standards are being followed.

Key responsibilities

• To support development and maintenance of Hadoop solutions for the Enterprise.
• To be part of initiatives that brings data into the data lake and delivers insights.
• To perform production support and maintenance of existing datasets in Hadoop