Cloudera Data Platform Engineer

Tech Aalto Pte ltd; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Tech Aalto Pte Ltd is seeking a Cloudera Data Platform Engineer to manage and troubleshoot Cloudera components in a Singapore-based environment. The role focuses on HDFS, YARN, HIVE, Spark, Impala, and Ranger, with emphasis on security and system optimization.

You will monitor with Cloudera Manager, Zabbix, Grafana, Splunk, and SyslogNG, and develop automation scripts in Bash and Python. Collaboration with Informatica and Denodo ensures smooth data integration and processing.

Qualifications

• Hands-on experience with at least three components of the Cloudera Data Platform: HDFS, YARN, HIVE, Spark, Impala, Ranger.
• Proficiency with monitoring tools like Cloudera Manager, Zabbix, Grafana, Splunk, and SyslogNG.
• Strong scripting skills such as Bash or Python for automation and process optimization.
• Familiarity with middleware applications like Informatica and Denodo.
• Basic understanding of operating systems, security, and networking configurations.
• Experience with cloud technologies (AWS, Azure) is a plus.

Responsibilities

• Manage and troubleshoot Cloudera Data Platform components (HDFS, YARN, HIVE, Spark, Impala, Ranger).
• Ensure data security and integrity and optimize performance across the data ecosystem.
• Develop and maintain automation scripts for provisioning, monitoring, and maintenance.
• Collaborate with middleware and integration tools (Informatica, Denodo) for data processing.

Skills

HDFS

YARN

HIVE

Spark

Impala

Ranger

Monitoring

Bash

Python

Security basics

Tools

Cloudera Manager

Zabbix

Grafana

Splunk

SyslogNG

Informatica

Denodo

AWS

Azure

Job description

We are looking for a skilled Cloudera Data Platform Engineer to join our dynamic team. The ideal candidate will have hands-on experience with Cloudera Data Platform components such as HDFS, YARN, HIVE, Spark, Impala, Ranger, along with strong knowledge of operating systems, security, and networking. You will be responsible for monitoring, troubleshooting, and optimizing big data environments using advanced tools and automation scripts.

Key Responsibilities

• Cloudera Data Platform Management:
  • Manage and troubleshoot components of theCloudera Data Platform, including at least three of the following:HDFS, YARN, HIVE, Spark, Impala, Ranger.
  • Ensure the security and integrity of data within the platform.
  • Optimize performance and resource utilization across the data ecosystem.
• Monitoring and Troubleshooting:
  • Utilize monitoring tools such asCloudera Manager, Zabbix, Grafana, Splunk, and SyslogNGfor proactive system monitoring and incident management.
  • Perform root cause analysis and implement corrective actions to ensure high availability and reliability of data systems.
• Automation and Scripting:
  • Develop and maintain automation scripts usingbash, Python, or shell scriptingto streamline operational tasks.
  • Implement automated solutions for system provisioning, monitoring, and maintenance.
• Middleware and Integration:
  • Collaborate with middleware applications such asInformaticaandDenodoto ensure seamless data integration and processing.
  • Ensure compatibility and integration with other enterprise systems.
• Cloud Technology Exposure (Optional):
  • Work withcloud technologies (AWS, Azure)for data storage, processing, and migration (if applicable).
  • Collaborate with cloud architects to optimize data solutions in hybrid cloud environments.

Requirements

• Technical Skills:
  • Hands-on experience with at leastthree componentsof theCloudera Data Platform:HDFS, YARN, HIVE, Spark, Impala, Ranger.
  • Proficiency withmonitoring toolslikeCloudera Manager, Zabbix, Grafana, Splunk, and SyslogNG.
  • Strong skills inscripting languages such asbash, Python, or shell scriptingfor automation and process optimization.
  • Familiarity withmiddleware applicationslikeInformaticaandDenodo.
  • Basic understanding ofoperating systems, security, and networkconfigurations.
• Experience:
  • [Specify required years of experience, e.g., 3-5 years] in managing and troubleshooting Cloudera Data Platform environments.
  • Experience inmonitoring and incident managementusing advanced monitoring tools.
  • Knowledge ofcloud technologies (AWS, Azure)is a plus.
• Soft Skills:
  • Strong problem-solving and analytical skills.
  • Excellent communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.
  • Ability to manage multiple tasks and prioritize effectively in a fast-paced environment.

Preferred Qualifications

• Certification inCloudera Data Platformor related big data technologies.
• Experience withcloud technologies (AWS, Azure)for data storage and processing.
• Familiarity withbig data frameworksand ecosystems.