Assistant Manager - Big Data Administrator.MGN Pak - Digital Data Platforms.MIT - PAKCOE

Mashreq; Karachi Division; On-site; PKR 2,009,000 - 2,902,000; Full time

Job summary

Mashreq is seeking an experienced Cloudera Administrator to manage and support our enterprise Hadoop ecosystem. The role focuses on stability, performance, and security across CDP/CDH clusters, with involvement in data ingestion, processing, and analytics workloads.

The ideal candidate has 4–7 years of hands-on Cloudera/Hadoop experience, strong Linux skills, and the ability to automate tasks, upgrade clusters, and collaborate with data engineers to optimize pipelines.

Qualifications

• Bachelor’s degree or equivalent experience in CS/IT.
• 4–7 years of hands-on Cloudera/Hadoop administration.
• Strong Linux/Unix administration and shell scripting.
• Experience with Kerberos, Ranger, LDAP/AD for security.
• Knowledge of cloud deployments (AWS/Azure/GCP) a plus.

Responsibilities

• Install, configure, and maintain CDP/CDH clusters in prod and non-prod.
• Monitor cluster health, performance, and resource utilization.
• Manage HDFS, YARN, Hive, Impala, Spark, HBase, Kafka, and other components.
• Perform upgrades, patches, migrations with minimal downtime.
• Implement security controls with Kerberos, Ranger, LDAP/AD.
• Automate tasks using Python, Shell, or Ansible.
• Troubleshoot issues, bottlenecks, and job failures.
• Support backup and disaster recovery planning.
• Collaborate with Data Engineers and Architects to optimize pipelines.
• Generate usage reports and capacity planning insights.

Skills

Cloudera/Hadoop

Linux admin

Shell scripting

Cluster monitoring

Security: Kerberos

LDAP/AD integration

Automation: Python/Ansible

Upgrades & performance tuning

Education

Bachelor’s degree in Computer Science/IT or equivalent

Tools

Cloudera Manager

Job description

Job Description: Cloudera Administrator

Role: Assistant Manager ( J+)

Position Summary

We are seeking an experienced Cloudera Administrator with 4 to 7 years of experience to manage, configure, and support our enterprise-grade Cloudera Hadoop ecosystem. This role involves ensuring the stability, performance, and security of Cloudera clusters, along with supporting data ingestion, processing, and analytics workloads. The ideal candidate will have strong expertise in Cloudera/HDP/CDP distributions, Linux system administration, and enterprise big data platforms.

Key Responsibilities

• - Install, configure, and maintain Cloudera Data Platform (CDP) or Cloudera Distribution of Hadoop (CDH) clusters in production and non-production environments.
• - Monitor cluster health, performance, and resource utilization, ensuring high availability and reliability.
• - Manage and support Hadoop ecosystem components (HDFS, YARN, Hive, Impala, Spark, HBase, Kafka, Oozie, Zookeeper, Ranger, Atlas, etc.).
• - Perform cluster upgrades, patches, and migrations with minimal downtime.
• - Implement and maintain security controls using Kerberos, Ranger, and LDAP/AD integration.
• - Manage user access, roles, and privileges across the Cloudera ecosystem.
• - Automate routine administrative tasks using Python, Shell scripting, or Ansible.
• - Troubleshoot cluster-related issues, performance bottlenecks, and job failures.
• - Support backup and disaster recovery planning for Hadoop clusters.
• - Collaborate with Data Engineers, Developers, and Architects to optimize data pipelines and workflows.
• - Generate reports on cluster usage, job performance, and capacity planning.

Required Skills & Qualifications

• - Bachelor’s degree in computer science, Information Technology, or related field (or equivalent experience).
• - 4 to 7 years of hands-on experience administering Cloudera/Hadoop ecosystems.
• - Strong experience with Linux/Unix system administration and shell scripting.
• - Expertise in Hadoop ecosystem components (HDFS, YARN, Hive, Spark, Impala, Kafka, HBase).
• - Knowledge of Cloudera Manager for monitoring, configuration, and cluster administration.
• - Familiarity with Kerberos, Ranger, Sentry, and LDAP/AD integration for security.
• - Proficiency in troubleshooting, log analysis, and performance tuning.
• - Experience in backup, recovery, and disaster recovery strategies for big data platforms.
• - Knowledge of cloud platforms (AWS, Azure, GCP) and cloud-based Cloudera deployments is a plus.

Preferred Qualifications

• - Cloudera Certified Administrator for Apache Hadoop (CCAH) or equivalent certification.
• - Familiarity with containerization (Docker, Kubernetes) for big data workloads.
• - Knowledge of SQL, Python, PySpark for debugging data pipelines.

Soft Skills

• - Strong problem-solving and analytical skills.
• - Ability to work collaboratively with cross-functional teams.
• - Excellent communication and documentation skills.
• - Proactive and self-motivated with a focus on continuous improvement.

Key results area -

1. 1. Cluster Management & Stability
   • - Ensure high availability and optimal performance of Cloudera clusters.
   • - Maintain uptime and reliability across production and non-production environments.
   • - Perform timely upgrades, patches, and migrations with minimal disruption.
2. 2. Hadoop Ecosystem Administration
   • - Efficient management of core components: HDFS, YARN, Hive, Impala, Spark, Kafka, HBase, etc.
   • - Monitor and troubleshoot ecosystem services and jobs.
3. 3. Security & Access Control
   • - Implement and maintain robust security using Kerberos, Ranger, and LDAP/AD.
   • - Manage user roles, access privileges, and audit logs.
4. 4. Automation & Scripting
   • - Automate administrative tasks using Python, Shell, or Ansible.
   • - Reduce manual intervention and improve operational efficiency.
5. 5. Performance Monitoring & Optimization
   • - Monitor cluster health and resource utilization.
   • - Identify and resolve performance bottlenecks in data processing jobs.
6. 6. Backup & Disaster Recovery
   • - Develop and maintain backup strategies and disaster recovery plans.
   • - Ensure data integrity and availability during failures.
7. 7. Collaboration & Support
   • - Work closely with Data Engineers, Developers, and Architects to support data pipelines.
   • - Provide technical guidance and support for analytics workloads.
8. 8. Reporting & Documentation
   • - Generate usage reports, performance metrics, and capacity planning insights.
   • - Maintain clear documentation for configurations, procedures, and troubleshooting.
9. 9. Compliance & Governance
   • - Ensure adherence to data governance policies using tools like Atlas and Ranger.
   • - Support audits and compliance checks.
10. 10. Continuous Improvement & Learning
    • - Stay updated with Cloudera/CDP advancements and best practices.
    • - Proactively suggest improvements in architecture and operations.