Hadoop Administrator

Clover Infotech; Bengaluru; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Clover Infotech is seeking an experienced Senior Big Data Engineer to join our team in Bengaluru. You will manage Tier 2 production environments, install and maintain Cloudera CDP, and optimize Spark and Hive jobs for high availability across global deployments.

You will collaborate with data engineers, implement CI/CD pipelines, and automate security and patching tasks. Strong scripting in Shell/Terraform/Groovy and container tech (Docker/Kubernetes/OpenShift) are essential.

Qualifications

• At least 4 years production Hadoop experience.
• 8+ years hands on Technology experience.
• Experience with large and complex global enterprises defined by high availability, broad observability, and high level of reliability.
• At least 1 project experience in Data Science project/products.
• Excellent communication and articulation skills.

Responsibilities

• Hands on Tier 2 production experience with Cloudera:
• Installing and maintenance
• Enabling security requirements for CDP
• Optimizing cluster performance
• Working with data engineers to optimize Spark job performance
• Automation – Ansible, data/analytics jobs CI/CD
• Observability
• Installation, configuration and maintenance of the CDP latest version of CDP 7.1.9 or CDP latest version experience with Cloudera data services 1.5.4 and 1.5.5
• Trouble shooting the Spark job, Hive Jobs and Yarn jobs
• CDP Containerization with data services
• experience in writing automation using Shell, Terraform, Groovy and Ansible.
• Automated pipelines/workflows (GitHub Actions, Jenkins, ArgoCD etc.) for data projects. Python based preferred.
• Automating security/regulatory checks
• Automated patching
• Containerization – Podman, Docker, OCP ,
• Container Management – Kubernetes/OpenShift
• Linux OS & Networking basics
• Source control (git)
• Agile engineering and practices

Skills

Cloudera CDP

Spark tuning

CI/CD pipelines

Ansible

Terraform

Groovy

Python

Kubernetes

OpenShift

Docker

Shell scripting

Git

Tools

Hadoop

Cloudera CDP

CDP Data Services

Ansible

Terraform

Groovy

Python

GitHub Actions

Jenkins

ArgoCD

Podman

Docker

Kubernetes

OpenShift

Shell scripting

Job description

Job Descriptions:

• Hands on Tier 2 production experience with Cloudera:
• Installing and maintenance
• Enabling security requirements for CDP
• Optimizing cluster performance
• Working with data engineers to optimize Spark job performance
• Automation – Ansible, data/analytics jobs CI/CD
• Observability
• Installation, configuration and maintenance of the CDP latest version of CDP 7.1.9 or CDP latest version experience with Cloudera data services 1.5.4 and 1.5.5
• Trouble shooting the Spark job , Hive Jobs and Yarn jobs
• CDP Containerization with data services
• experience in writing automation using Shell, Terraform, Groovy and Ansible.
• Automated pipelines/workflows (GitHub Actions, Jenkins, ArgoCD etc.) for data projects. Python based preferred.
• Automating security/regulatory checks
• Automated patching
• Containerization – Podman, Docker, OCP ,
• Container Management – Kubernetes/OpenShift
• Linux OS & Networking basics
• Source control (git)
• Agile engineering and practices

Required Experience:

• At least 4 years production Hadoop experience
• 8+ years hands on Technology experience
• Experience with large and complex global enterprises defined by high availability, broad observability, and high level of reliability.
• At least 1 project experience in Data Science project/products
• Excellent communication and articulation skills