Data Engineer

User Experience Researchers Pte.Ltd; Singapore; On-site; SGD 75,000 - 95,000; Full time

Job summary

A leading data solutions provider in Singapore seeks a skilled operations manager to oversee day-to-day operations of a big data platform. The ideal candidate will possess hands-on experience with Cloudera Data Platform and strong troubleshooting skills. Key responsibilities include monitoring platform performance, addressing security vulnerabilities, and developing automation scripts. Candidates should demonstrate excellent communication abilities and the capacity to handle high-pressure situations. This role emphasizes maintaining high availability and reliability of IT solutions.

Qualifications

• Hands-on experience with Cloudera Data Platform including HDFS, YARN, HIVE, Spark, and Impala.
• Experience with monitoring tools like Cloudera Manager, Zabbix, and Grafana.
• Ability to troubleshoot complex issues from system resources to application stacks.

Responsibilities

• Oversee operations of the big data platform to ensure high availability.
• Monitor platform services and identify potential issues.
• Develop automation scripts to streamline tasks and ensure operational consistency.

Skills

Proactive monitoring

Documentation skills

Automation skills

Cloud technology experience

High availability and performance

Education

Cloudera Certified Administrator

Tools

Cloudera Data Platform

Cloudera Manager

Zabbix

Grafana

Splunk

AWS

Azure

Job description

Responsibilities

• Oversee day-to-day operations of big data platform to ensure high availability, reliability and performance.
• Proactively monitor big data platform services, components and clusters to identify potential issues. Take corrective actions as needed to maintain platform health
• Manage configuration, upgrades, and patching of big data platform, ensuring all services are up to date
• Work with the clients technical teams to ensure smooth deployment and adoption of new solution to support data ingestions, process and workflows
• Maintain clear and detailed documentation of platform configuration, troubleshooting steps and incident resolution.
• Continuously monitor for and address platform security vulnerabilities. Implement patching strategies to resolve identified vulnerabilities and maintain a secure environment.
• Develop automation script to streamline administrative tasks, platform health and ensure operational consistency.
• Ensure the smooth operations and service level of IT solutions.
• Support production issues

Qualifications

• Hands-on experience, knowledge and troubleshooting of Cloudera Data Platform such as HDFS, YARN, HIVE, Spark, Impala, Ranger, operating systems, security and network.
• Hands on experience with monitoring tools like Cloudera Manager, Zabbix, Grafana, Splunk, SyslogNG
• Familiarity with middleware applications i.e. Informatica, Denodo and scripting languages like bash, python, or shell scripting for automation
• Experience with cloud technology i.e. AWS, Azure is a plus
• Ability to troubleshoot complex issues ranging from system resource to application stack traces.
• Track record in implementing systems with high availability, high performance, high security hosted at various data centres or hybrid cloud environments will be an added advantage.
• Cloudera Certified Administrator or similar certification are a plus.
• Excellent communication skill to work with cross-functional teams
• Ability to handle high-pressure situations and manage critical incidents