Senior Hadoop Administrator

Smart Analytica; Mumbai; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Smart Analytica in Mumbai is seeking an experienced CDP Hadoop Administrator - Lead to manage and support our Hadoop ecosystem. The ideal candidate will have strong expertise in Hadoop cluster administration, troubleshooting, and a proven track record of maintaining and optimizing large data environments.

You will install, configure, monitor, and tune Hadoop components, ensure high availability, security, and disaster recovery, and collaborate with development teams to translate requirements

Qualifications

• BE / B.Tech in Computer Science or equivalent.
• Experience in installing, configuring and tuning Hadoop distributions.
• Hands-on experience with Cloudera.
• Knowledge of ETL concepts and data management fundamentals.

Responsibilities

• Install, configure, and manage Hadoop clusters (HDFS, YARN, Hive, HBase).
• Monitor performance, capacity, and security of clusters.
• Perform upgrades, patching, backups, and DR planning.
• Implement data ingestion with Sqoop, Flume, and Kafka.
• Ensure HA and disaster recovery for Hadoop environments.
• Collaborate with development teams to understand requirements.
• Create and maintain cluster documentation.

Skills

Hadoop

Cloudera

Big data concepts

Scripting (Shell/Python)

Education

BE / B.Tech in Computer Science

Tools

Cloudera

Hadoop

HDFS

YARN

Hive

HBase

Sqoop

Flume

Kafka

Ganglia

Nagios

Shell

Python

Perl

Job description

Role & responsibilities

Role: CDP Hadoop Admin - Lead

Experience 7+ years

Location - BKC, Mumbai

Work mode - 5 Days Work from Office (FTE)

Shift: Rotational(24*7) & 2 Rotational Week offs

Job Summary:

We are seeking an experienced CDP HadoopAdministrator - Lead to manage and support our Hadoopecosystem. The ideal candidate will have strong expertise in Hadoopcluster administration, excellent troubleshooting skills, and a proven track record of maintaining and optimizing Hadoopenvironments.

KeyResponsibilities:

• Install, configure, and manage Hadoopclusters, including HDFS, YARN, Hive, HBase, and other ecosystem components.
• Monitor and manage Hadoopcluster performance, capacity, and security.
• Perform routine maintenance tasks such as upgrades, patching, and backups.
• Implement and maintain data ingestion processes using tools like Sqoop, Flume, and Kafka.
• Ensure high availability and disaster recovery of Hadoopclusters.
• Collaborate with development teams to understand requirements and provide appropriate Hadoopsolutions.
• Troubleshoot and resolve issues related to the Hadoopecosystem.
• Maintain documentation of Hadoopenvironment configurations, processes, and procedures.

Requirement:

• Experience in Installing, configuring and tuning Hadoopdistributions.
• Hands on experience in Cloudera.
• Understanding of Hadoopdesign principals and factors that affect distributed system performance, including hardware and network considerations.
• Provide Infrastructure Recommendations, Capacity Planning, work load management.
• Develop utilities to monitor cluster better Ganglia, Nagios etc.
• Manage large clusters with huge volumes of data
• Perform Cluster maintenance tasks
• Create and removal of nodes, cluster monitoring and troubleshooting
• Manage and review Hadooplog files
• Install and implement security for Hadoopclusters
• Install HadoopUpdates, patches and version upgrades. Automate the same through scripts
• Point of Contact for Vendor escalation. Work with Hortonworks in resolving issues
• Should have Conceptual/working knowledge of basic data management concepts like ETL, Ref/Master data, Data quality, RDBMS
• Working knowledge of any scripting language like Shell, Python, Perl
• Should have experience in Orchestration & Deployment tools.

Academic Qualification:

• BE / B.Tech in Computer Science or equivalent along with hands-on experience in dealing with large data sets and distributed computing in data warehousing and business intelligence systems using Hadoop.