Hadoop Administrator

Home Credit; Gurugram District; On-site; INR 2,000,000 - 3,000,000; Full time

Job summary

Home Credit seeks an experienced Platform Operations & Administration professional to manage Hadoop ecosystems, ensure platform stability, and drive continuous improvement across clusters in dev/test/prod environments.

The role focuses on provisioning, upgrades, incident management, capacity planning, and reliability engineering to meet SLA targets and ensure data durability. 5 days/week on-site at Gurugram.

Qualifications

• Experience administering Hadoop ecosystem clusters and related services.
• Ability to plan and execute upgrades, patches, and migrations with minimal downtime.
• Strong monitoring, incident management and RCA skills.
• Experience with capacity planning, HA, DR readiness, and reliability practices.

Responsibilities

• Administer and support Hadoop ecosystem clusters (HDFS, YARN, MapReduce) to meet SLA targets.
• Provision, configure, tune, and perform day-to-day operational support and health checks.
• Manage multi-cluster environments and standardize configurations and procedures.
• Plan platform upgrades, patching, and migrations with minimal downtime.
• Coordinate releases and manage vendor distributions (Cloudera/HDP/Apache).
• Implement robust monitoring/alerting and drive RCAs and corrective actions.
• Tune HDFS/YARN and JVM parameters to optimize performance and stability.
• Forecast growth and perform capacity planning for storage and compute.
• Ensure data durability and operational resilience through maintenance and DR drills.

Skills

Hadoop Admin

Cluster Operations

HDFS/YARN

Upgrade & Patch Management

Monitoring & Incident Mgmt

Performance Tuning

Capacity Planning

Tools

Cloudera/HDP/Apache

Job description

Job Description :

Platform Operations & Administration

• Administer and support Hadoop ecosystem clusters (e.g., HDFS, YARN, MapReduce) and related services to meet SLA/SLI targets for availability and performance.
• Perform cluster provisioning, configuration, tuning, and day-to-day operational support (start/stop, health checks, service validation).
• Manage multi-cluster environments (dev/test/prod), including standardization of configurations and operational procedures.

Installation, Upgrades & Lifecycle Management

• Plan and execute platform upgrades, patching, and migrations with minimal downtime (including rolling upgrades where applicable).
• Maintain platform compatibility matrix, validate integrations, and coordinate release rollouts with stakeholders.
• Manage vendor distributions and tooling (e.g., Cloudera/HDP/Apache), including repository, parcels/packages, and version control.

Monitoring, Incident & Problem Management

• Implement robust monitoring/alerting for platform and infrastructure (CPU, memory, disk, network, I/O, JVM, services, queues, HDFS capacity, GC, etc.).
• Lead troubleshooting for incidents, perform root cause analysis (RCA), and drive corrective/preventive actions.
• Own operational runbooks, on-call readiness, escalation processes, and continuous improvement of response/MTTR.

Performance Tuning & Optimization

• Tune HDFS/YARN for workload performance and stability (scheduler/queue policies, resource limits, container sizing, node settings).
• Optimize cluster performance through JVM tuning, service parameters, OS/kernel tuning, and workload analysis.
• Identify and remediate bottlenecks (skew, small files, disk hotspots, network saturation, metadata pressure, etc.)

Capacity Planning & Reliability Engineering

• Forecast growth and perform capacity planning for storage, compute, and service scaling.
• Drive high availability and disaster recovery readiness (NameNode HA, ZK quorum health, backup/restore, replication, snapshots, DR drills).
• Ensure data durability and operational resilience through proactive maintenance and reliability reviews.

We are 5 days work from office