Big Data Administrator

Capgemini; Bengaluru, Hyderabad; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Capgemini in Bengaluru is seeking an experienced Hadoop Administrator to manage and optimize a large-scale big data infrastructure. You will be responsible for cluster administration, performance tuning, security, and ensuring high availability across environments.

You will install and maintain Hadoop clusters, monitor health and capacity, and collaborate with data engineering, DevOps, and application teams to support workloads.

Qualifications

• Hands-on experience administering Hadoop clusters (CDH/CDP, HDP, or Apache).
• Deep knowledge of HDFS, YARN, MapReduce, and Hadoop ecosystem.
• Experience with cluster security (Kerberos, Ranger/Sentry, TLS/SSL).
• Proficient in Linux/Unix administration.
• Scripting skills in Shell, Python, or Perl.

Responsibilities

• Install, configure, and maintain Hadoop clusters across dev, test, and production environments.
• Monitor cluster health, performance, capacity, and resource utilization; troubleshoot proactively.
• Manage HDFS, YARN, and related components (Hive, HBase, Spark, Sqoop, Oozie, Kafka, Zookeeper).
• Implement and manage cluster security, isolation, and access controls.
• Plan capacity, scale clusters, and handle decommissioning of nodes.

Skills

Hadoop clusters

HDFS / YARN

Security Kerberos

Linux administration

Shell / Python scripting

Monitoring tools

Networking & performance tuning

Tools

Cloudera Manager

Ambari

Grafana

Job description

We are looking for an experienced Hadoop Administrator to manage, maintain, and optimize our big data infrastructure. The ideal candidate will be responsible for cluster administration, performance tuning, security, and ensuring high availability of Hadoop ecosystems.

Key Responsibilities

• Install, configure, and maintain Hadoop clusters (Cloudera / Hortonworks / Apache) across development, test, and production environments.
• Monitor cluster health, performance, capacity, and resource utilization; troubleshoot and resolve issues proactively.
• Manage HDFS, YARN, and related ecosystem components (Hive, HBase, Spark, Sqoop, Oozie, Kafka, Zookeeper).
• Implement and manage cluster security using Kerberos, Ranger, Sentry, and LDAP integration.
• Perform capacity planning, cluster scaling, commissioning, and decommissioning of nodes.
• Configure high availability, backup, disaster recovery, and data replication strategies.
• Automate routine administration tasks using shell scripting, Python, or Ansible.
• Apply patches, upgrades, and version migrations with minimal downtime.
• Manage user access, quotas, and multi-tenancy across the cluster.
• Collaborate with data engineering, DevOps, and application teams to support workloads.

Required Skills

• Strong hands‑on experience administering Hadoop clusters (Cloudera CDH/CDP, Hortonworks HDP, or Apache Hadoop).
• In‑depth knowledge of HDFS, YARN, MapReduce, and the broader Hadoop ecosystem.
• Experience with cluster security (Kerberos, Ranger/Sentry, TLS/SSL).
• Proficiency in Linux/Unix administration.
• Scripting skills in Shell, Python, or Perl.
• Familiarity with monitoring tools (Cloudera Manager, Ambari, Grafana, Nagios).
• Understanding of networking, storage, and performance tuning concepts.