Hadoop Administrator

Infosys; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Infosys in Bengaluru is seeking a Hadoop Administrator to manage large-scale Hadoop ecosystems, covering Cloudera, Hortonworks, or Apache Hadoop. You will optimize performance and collaborate with data engineers to ensure reliable processing.

The role emphasizes Linux administration, HA/DR, and hands-on experience with HDFS, YARN, and MapReduce within the Hadoop ecosystem.

Qualifications

• 3–6 years of experience in Hadoop Administration.
• Strong knowledge of HDFS, YARN, MapReduce and Hadoop ecosystem tools.
• Experience with Linux system administration (mandatory).
• Familiarity with Hive, Spark, HBase, Kafka, Sqoop, Flume.
• Experience with cluster installation, configuration, scaling, upgrades, and patching.

Responsibilities

• Install, configure, and maintain Hadoop clusters (Cloudera/Hortonworks/Apache).
• Manage core components: HDFS, YARN, MapReduce.
• Monitor cluster performance, health, and usage; troubleshoot issues.

Skills

Big Data

Hadoop Administration

Cloud Hadoop

DevOps (CI/CD, Docker, Kubernetes)

HA/DR setups

Tools

Cloudera Manager

Ambari

Grafana

Nagios

Prometheus

Shell/Python

Job description

Hadoop Admin

Primary skills

• Technology->Big Data
• Hadoop->Hadoop Administration
• Experience with cloud-based Hadoop (AWS EMR, Azure HDInsight, GCP Dataproc)
• Knowledge of DevOps practices (CI/CD, Docker, Kubernetes)
• Experience in high availability (HA) and disaster recovery (DR) setups

Cluster Management

• Install, configure, and maintain Hadoop clusters (Cloudera, Hortonworks, or Apache Hadoop)
• Manage core components including HDFS, YARN, MapReduce
• Perform cluster scaling, upgrades, and patching

Administration & Monitoring

• Monitor cluster performance, health, and usage
• Troubleshoot and resolve cluster issues, failures, and bottlenecks
• Implement alerting and monitoring tools (Ambari, Cloudera Manager, Grafana)

Security & Governance

• Configure and manage Kerberos authentication
• Implement data security policies using Ranger / Sentry
• Handle user access controls and permissions

Data Management

• Manage data storage, replication, and backup strategies
• Optimize HDFS performance and storage utilization
• Handle data ingestion and integration workflows

Performance Optimization

• Tune Hadoop cluster performance (memory, CPU, storage)
• Optimize YARN resource allocation and scheduling
• Identify and resolve performance issues in jobs

Integration & Support

• Support tools in Hadoop ecosystem: Hive Spark HBase Sqoop / Flume / Kafka
• Work closely with data engineers and developers

Required Skills & Qualifications

Core Technical Skills

• 3-6 years of experience in Hadoop Administration
• Strong knowledge of: HDFS, YARN, MapReduce Hadoop ecosystem tools
• Experience with Linux system administration (mandatory)

Big Data Ecosystem

• Familiarity with: Hive, Spark, HBase Kafka, Sqoop, Flume
• Understanding of distributed computing concepts

Tools & Platforms

• Experience with: Cloudera Manager / Ambari Monitoring tools (Nagios, Prometheus, Grafana)
• Basic scripting in Shell / Python