IT Storage Engineer

MTS SYSTEMS ENGINEERING PTE. LTD.; Singapore; On-site; SGD 70,000 - 110,000; Full time

Job summary

MTS SYSTEMS ENGINEERING PTE. LTD. is seeking an experienced Hadoop/HDFS administrator to design, deploy and maintain enterprise-scale HDFS clusters.

You will monitor performance, manage capacity, implement security controls, and troubleshoot data-node issues to ensure high availability. Responsibilities include data ingestion optimization, collaboration with data engineering teams, and automating operations with Bash and Python.

Qualifications

• Bachelor’s degree in Computer Science, Information Systems, or related field.
• 3–5+ years of experience with Hadoop ecosystem, especially HDFS administration.
• Strong understanding of HDFS architecture, replication, and fault tolerance.
• Experience with Cloudera, Hortonworks, or Apache Hadoop distributions.
• Proficiency in Linux/Unix system administration and scripting (Bash, Python).
• Familiarity with YARN, Hive, HBase, Spark, Oozie, and Zookeeper.
• Experience with monitoring tools like Ambari, Cloudera Manager, or Nagios.

Responsibilities

• Design, deploy, and maintain HDFS clusters to support enterprise-scale data applications.
• Monitor cluster performance, manage storage capacity, and ensure high availability and fault tolerance.
• Implement data security, access controls, and encryption for HDFS data.
• Troubleshoot and resolve issues related to HDFS, including data node failures, replication issues, and performance bottlenecks.
• Manage data ingestion pipelines and optimize data storage formats (e.g. Parquet, Avro).
• Support and work with data engineering and analytics teams to ensure reliable data delivery and transformation workflows.
• Automate cluster operations using scripting (e.g., Bash, Python) and orchestration tools.
• Conduct upgrades and patching of Hadoop ecosystem components (HDFS, YARN, Hive, etc.).
• Maintain documentation of architecture, configurations, and best practices.
• Ensure compliance with data governance and data privacy policies.

Skills

Hadoop ecosystem experience

HDFS administration

Linux/Unix scripting

Monitoring tools

Education

Bachelor's degree in CS/IS

Tools

Ambari

Cloudera Manager

Nagios

Hadoop distributions

Docker

Kubernetes

Oozie

Zookeeper

YARN

Hive

HBase

Spark

Job description

Key Responsibilities:

1. 1. Design, deploy, and maintain HDFS clusters to support enterprise-scale data applications.
2. Monitor cluster performance, manage storage capacity, and ensure high availability and fault tolerance.
3. Implement data security, access controls, and encryption for HDFS data.
4. Troubleshoot and resolve issues related to HDFS, including data node failures, replication issues, and performance bottlenecks.
5. Manage data ingestion pipelines and optimize data storage formats (e.g. Parquet, Avro).
6. Support and work with data engineering and analytics teams to ensure reliable data delivery and transformation workflows.
7. Automate cluster operations using scripting (e.g., Bash, Python) and orchestration tools.
8. Conduct upgrades and patching of Hadoop ecosystem components (HDFS, YARN, Hive, etc.).
9. Maintain documentation of architecture, configurations, and best practices.
10. Ensure compliance with data governance and data privacy policies.

Qualifications and Requirement:

• a) Bachelor’s degree in Computer Science, Information Systems, or a related field.
• 3–5+ years of experience with Hadoop ecosystem, particularly HDFS administration.
• Strong understanding of HDFS architecture, replication, and fault tolerance.
• Experience with Cloudera, Hortonworks, or Apache Hadoop distributions.
• Proficiency in Linux/Unix system administration and scripting (Bash, Python, etc.).
• Familiarity with related components: YARN, Hive, HBase, Spark, Oozie, and Zookeeper.
• Experience with monitoring tools like Ambari, Cloudera Manager, or Nagios.

Advantage to have: -

• Hadoop certification (e.g., Cloudera Certified Administratorfor Apache Hadoop - CCAH).
• Knowledge of cloud-based big data platforms (AWS EMR, Azure HDInsight, GCPDataproc).
• Experience with containerization (Docker/Kubernetes) for big data workloads.
• Exposure to data lake architectures and data governance tools.