Hadoop Administrator

Capgemini; Bengaluru; On-site; INR 800,000 - 1,200,000; Full time

Job summary

Capgemini is looking for an experienced Hadoop Administrator in Bengaluru, India, to manage and optimize our big data infrastructure. This role includes responsibilities like deploying and maintaining Hadoop clusters, monitoring performance, and ensuring security through frameworks like Kerberos.

The ideal candidate will have hands-on experience with Hadoop distributions, strong Linux administration skills, and scripting expertise. Join us to work alongside data engineering and DevOps teams to support data pipelines in a dynamic environment.

Qualifications

• Hands-on experience administering Hadoop distributions like Cloudera or Hortonworks.
• Strong knowledge of HDFS, YARN, and the broader Hadoop ecosystem.
• Proficiency in Linux/Unix administration (RHEL, CentOS, or Ubuntu).
• Experience with Kerberos and cluster security frameworks.
• Scripting skills in Shell, Python, or Bash.
• Familiarity with monitoring and automation tools.

Responsibilities

• Install, configure, and maintain Hadoop clusters across different environments.
• Manage cluster capacity planning and scaling.
• Monitor cluster health and performance using various tools.
• Implement and manage security via frameworks like Kerberos.
• Perform backup and recovery for HDFS data.
• Troubleshoot and resolve cluster-related issues.
• Administer ecosystem components such as Kafka and Zookeeper.
• Automate routine tasks using scripts.
• Collaborate with data engineering and DevOps teams.

Skills

Hadoop distributions administration

HDFS

YARN

Linux/Unix administration

Scripting skills

Tools

Ambari

Cloudera Manager

Hive

HBase

Spark

Ansible

Job description

We are seeking an experienced Hadoop Administrator to manage, maintain, and optimize our big data infrastructure. The ideal candidate will be responsible for the deployment, configuration, monitoring, and performance tuning of Hadoop clusters in a production environment.

Key Responsibilities:

• Install, configure, and maintain Hadoop clusters (HDFS, YARN, MapReduce) across development, test, and production environments.
• Manage cluster capacity planning, scaling, commissioning, and decommissioning of nodes.
• Monitor cluster health, performance, and resource utilization using tools such as Ambari, Cloudera Manager, or Ganglia/Nagios.
• Implement and manage security via Kerberos, Ranger, Knox, and LDAP integration.
• Perform backup, recovery, and disaster-recovery planning for HDFS data.
• Troubleshoot and resolve cluster-related issues, job failures, and performance bottlenecks.
• Administer ecosystem components: Hive, HBase, Spark, Sqoop, Oozie, Kafka, and Zookeeper.
• Apply patches, upgrades, and routine maintenance with minimal downtime.
• Automate routine administrative tasks using shell scripting, Python, or Ansible.
• Collaborate with data engineering and DevOps teams to support data pipelines.

Required Skills:

• Hands‑on experience administering Hadoop distributions (Cloudera CDH/CDP, Hortonworks HDP, or Apache Hadoop).
• Strong knowledge of HDFS, YARN, and the broader Hadoop ecosystem.
• Proficiency in Linux/Unix administration (RHEL, CentOS, or Ubuntu).
• Experience with Kerberos and cluster security frameworks.
• Scripting skills in Shell, Python, or Bash.
• Familiarity with monitoring and automation tools.