Cloudera Platform Engineer

ValueLabs; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

A technology consulting firm is seeking a Lead Talent Acquisition Specialist in Bengaluru to build high-performing teams and craft innovative hiring strategies. The ideal candidate will have over 6 years of experience in information technology, with expertise in managing Hadoop and Cloudera environments. Responsibilities include deploying clusters, optimizing performance, and collaborating with data teams. This full-time position requires strong communication and problem-solving skills, along with a relevant degree and certification in Cloudera administration.

Qualifications

• 6+ years of experience in information technology.
• 3-5 years managing Hadoop/Cloudera environments.
• Cloudera Certified Administrator for Apache Hadoop (CCAH) or similar certification.

Responsibilities

• Deploy and manage Cloudera clusters.
• Optimize cluster performance and conduct regular testing.
• Collaborate with teams to understand data requirements.

Skills

Hadoop ecosystem components

Cloudera CDP

Linux system administration

Scripting languages (Python, Bash)

Data security practices

DevOps practices

Communication skills

Problem-solving skills

Education

Bachelor's or master's degree in computer science or related field

Tools

Cloudera Manager

Ansible

Docker

Kubernetes

Jenkins

Job description

Lead Talent Acquisition Specialist @Valuelabs | Building High-Performing Teams & Crafting Innovative Hiring Strategies.

Education - master’s or bachelor's degree in computer science, information systems management or related field.

Preferred Experience: More than 6+ years of experience in information technology, Minimum of 3-5 years of experience in managing and administering Hadoop/Cloudera environments.

Proven experience with Cloudera CDP (Cloudera Data Platform), Cloudera Manager, and related tools.

Strong knowledge of Hadoop ecosystem components (HDFS, YARN, Hive, HBase, Spark, Impala, etc.).

Proficiency in Linux system administration with experience with scripting languages (Python, Bash, etc.) and configuration management tools (Ansible, Puppet, etc.).

Familiarity with data security practices and tools (Kerberos, Ranger, Sentry).

Excellent problem-solving and analytical with strong communication and interpersonal skills.

Ability to work independently and in a team environment.

Cloudera Certified Administrator for Apache Hadoop (CCAH) or similar certification. Experience with DevOps practices and tools (Docker, Kubernetes, Jenkins).

Cluster Management

Deploy Cloudera clusters from scratch, including hardware and software configuration.

Configure clusters to meet specific business requirements and workloads.

Plan and execute cluster upgrades to ensure systems are up to date with the latest features and security patches.

Perform necessary tests to ensure system stability post-upgrade.

Continuously monitor the health and performance of the clusters using Cloudera Manager and other monitoring tools.

Implement alerting mechanisms to proactively address issues before they impact performance.

System Administration

Manage Hadoop components such as HDFS, YARN, Hive, HBase, Spark, Impala, Kafka, and others.

Configure and manage data storage policies, replication, and fault tolerance.

Implement and manage security policies, including Kerberos for authentication and Ranger or Sentry for authorization.

Conduct regular security audits and compliance checks.

Develop and maintain scripts to automate routine administrative tasks (e.g., backups, log rotation, user management).

Use configuration management tools (Ansible etc.) to ensure consistent and repeatable cluster configurations.

Performance Optimization

Optimize cluster performance by tuning Hadoop configuration parameters and adjusting resource allocations.

Conduct regular performance testing and benchmarking.

Identify and resolve performance bottlenecks in the Hadoop ecosystem components.

Work with application developers to optimize data processing jobs for efficiency.

Best Practices Implementation

Establish and enforce best practices for data storage, retrieval, and processing.

Provide guidance on the optimal use of Hadoop ecosystem tools.

Collaboration and Support

Work closely with data engineers, data scientists, and analysts to understand data requirements and provide solutions.

Assist in the design and implementation of data models and storage strategies.

Provide Level 2 and Level 3 support for Cloudera platform-related issues.

Troubleshoot and resolve technical issues related to cluster operations promptly.

Documentation & Support

Maintain comprehensive documentation of the platform architecture, configurations, and standard operating procedures.

Document troubleshooting steps and resolutions for future reference.

Participate in an on-call rotation to provide after hours support as needed.

Ensure rapid response to critical issues affecting cluster performance or availability.

Communicate effectively with stakeholders regarding system status, issues, and planned enhancements.

Provide regular updates and reports on cluster performance and health.

Collaborate with cross-functional teams to understand data requirements and provide solutions.

Participate in planning and design sessions for new data initiatives.

Provide training and support to junior team members and other stakeholders.

Mentor team members on best practices and new technologies within the Cloudera ecosystem.

Behavioral Competencies

Behavioral Competencies • Thinking Related • People Related • Self Related

Business Acumens Organization leadership skills Coaching and mentoring Excellent analytical skills Demonstrate critical and systems of thinking ability. Ability to negotiate and influence Disciplined, organized Flexibility / Adaptability Visionary Performance driven

Seniority level

Mid-Senior level

Employment type

Full-time

Job function

Information Technology

Industries

IT Services and IT Consulting and Banking