Big Data Administrator - L3

CORNERSTONE GLOBAL PARTNERS PTE. LTD.; Singapore; On-site; SGD 120,000 - 160,000; Full time

Job summary

Cornerstone Global Partners Pte. Ltd. is seeking an experienced Big Data Administrator to join the IT infrastructure and production support team.

The role focuses on administration, monitoring, and optimisation of Hadoop-based Big Data platforms in a production environment. You will collaborate with infrastructure, development, architecture and project teams to ensure system availability and performance. Key responsibilities include building and maintaining Hadoop clusters, performing updates,

Qualifications

• Minimum 8 years of relevant experience in system administration, infrastructure support or a related technical environment.
• At least 1–2 years of hands-on experience deploying and administering large-scale Hadoop clusters.
• Bachelor's degree in Computer Science, Information Technology, Engineering or a related discipline.

Responsibilities

• Administer, maintain and support Hadoop-based Big Data infrastructure in production environments.
• Build, configure and maintain Hadoop clusters and associated components.
• Perform Hadoop updates, patches, upgrades and cluster deployments.
• Manage cluster nodes, including node addition, removal and configuration.
• Support and administer Kafka environments and troubleshoot related operational issues.

Skills

Linux admin

Hadoop admin

Kafka admin

OpenSearch admin

Elasticsearch admin

Python scripting

Shell scripting

Kerberos security

YARN / Hadoop

Education

Bachelor's degree in Computer Science / IT / Engineering

Tools

Ambari

Spark

Hive

Job description

We are looking for an experienced Big Data Administrator to join the IT infrastructure and production support team. The candidate will be responsible for the administration, maintenance, monitoring and optimisation of Big Data platforms and associated infrastructure in a production environment.This is an L3 technical role that will work closely with infrastructure, development, architecture and project teams to maintain system availability, stability, performance and operational readiness.

Key Responsibilities

• Administer, maintain and support Hadoop-based Big Data infrastructure in production environments.
• Build, configure and maintain Hadoop clusters and associated components.
• Perform Hadoop updates, patches, upgrades and cluster deployments.
• Manage cluster nodes, including node addition, removal and configuration.
• Support HDFS backup, restore, file-system management and monitoring activities.
• Perform Hadoop user administration, including Linux user setup, Kerberos authentication and access management.
• Monitor and troubleshoot Hadoop services using system logs, Hadoop logs and monitoring/alerting tools.
• Support and administer Kafka environments and troubleshoot related operational issues.
• Manage and monitor OpenSearch / Elasticsearch clusters.
• Perform capacity planning, performance tuning and operational optimisation of Big Data platforms.
• Monitor cluster connectivity, security and overall system health.
• Review technology changes and identify potential operational risks.
• Participate in the resolution of complex and high-severity incidents and drive issues through to resolution.
• Support production readiness activities for new technologies, projects and solutions.
• Work closely with development, architecture and project teams on technology roadmaps, implementations and transformation initiatives.
• Develop and maintain technical documentation, operational procedures and knowledge-base materials.
• Identify opportunities for service improvement, automation and operational efficiency.
• Develop and maintain shell, Perl or Python scripts for system administration and automation.
• Provide technical guidance and knowledge sharing to team members where required.
• Work collaboratively with cross-functional teams to maintain data availability, reliability and service quality.

Technical Requirements

Essential Skills

• Strong experience in Linux system administration .
• Hands-on experience with Hadoop administration , including HDFS, YARN and related ecosystem components.
• Experience with Redis cluster administration .
• Experience with MongoDB administration .
• Hands-on experience with Kafka administration and understanding of Kafka internals.
• Experience managing OpenSearch / Elasticsearch clusters .
• Strong scripting skills in at least one scripting language, preferably Shell, Perl or Python .
• Experience troubleshooting complex Big Data platform issues using system and application logs.
• Knowledge of system monitoring, performance tuning and capacity management.
• Knowledge of OS administration, including authentication, disk quotas, system limits and security patching.
• Ability to automate software installation, configuration and operational tasks on Linux servers.
• Experience working in large-scale infrastructure or data centre environments.

Good to Have

• Experience with Hadoop ecosystem components such as Hive, Pig, Oozie, HBase, Flume and Sqoop.
• Experience administering Hadoop clusters using YARN-based applications such as Spark.
• Experience with Apache Ambari or similar Hadoop management tools.
• Knowledge of PAM and Kerberos security mechanisms.
• Knowledge of hardware configuration, disk topology and RAID.
• Experience with virtual machine deployment and configuration.
• Knowledge of JVM administration and troubleshooting.
• Proficiency in Python, Java or Scala.
• Proficiency in SQL, Hive or other SQL-on-Hadoop technologies.
• Experience with ETL processes or related technologies.
• AWS cloud certification or relevant cloud experience.

Experience & Qualifications

• Minimum 8 years of relevant experience in system administration, infrastructure support or a related technical environment.
• At least 1-2 years of hands‑on experience deploying and administering large-scale Hadoop clusters.
• Bachelor's degree in Computer Science, Information Technology, Engineering or a related discipline.
• Experience in financial services or other highly regulated environments would be an advantage.
• Experience in software development using Java, Python, Perl, PHP, CSS or HTML would be an advantage.
• Experience with Spark development would be an advantage.

Other Requirements

• Strong analytical, troubleshooting and problem-solving abilities.
• Ability to work independently as well as collaboratively within a technical team.
• Ability to manage and resolve moderate to highly complex technical issues.
• Comfortable working in a production environment where timely incident res