Cloudera Admin

Intuition IT Solutions Ltd; Stratford-upon-Avon; Hybrid; GBP 70,000 - 90,000; Full time

Job summary

A leading technology solutions provider in the United Kingdom seeks a skilled professional to design and automate deployments of Cloudera CDP components. Candidates should have extensive experience with AWS services and the Cloudera ecosystem, along with strong skills in Terraform and CI/CD tools like GitHub Actions. The role demands scripting capabilities in Shell and Python, with a note that 3 days onsite work is required.

Qualifications

• Extensive experience with Cloudera cloud data platform (CDP) and knowledge of Big Data.
• Proficiency in Terraform for infrastructure as code.
• Strong hands-on experience with the Cloudera CDP and Hadoop ecosystem.
• Experience with CI/CD tools like GitHub Actions or similar.
• Solid scripting skills in Shell and Python.

Responsibilities

• Design and automate the deployment of Cloudera CDP components using Terraform.
• Build and maintain CI/CD pipelines using GitHub Actions.
• Collaborate to integrate CDP with existing workflows.
• Write automation scripts using Shell and Python.
• Manage infrastructure and configuration with YAML and JSON.

Skills

Cloudera CDP

Terraform

CI/CD pipelines

GitHub Actions

AWS

Shell scripting

Python

Hadoop ecosystem

DevOps

Tools

GitHub

CloudFormation

Job description

Responsibilities

• Design and automate the deployment of Cloudera CDP components (Data Lake, Data Hubs, Data Services) using Terraform.
• Build and maintain CI/CD pipelines using GitHub Actions for infrastructure and data pipeline automation
• Collaborate with data engineering teams to integrate CDP with existing Big Data workflows.
• Write and maintain automation scripts using Shell and Python.
• Manage infrastructure and configuration using YAML and JSON.
• Configuring and tuning Cloudera services for performance and security.
• Ensure secure, scalable, and cost-effective deployments on cloud platforms (AWS, Azure, or GCP).
• An understanding of all the Hadoop daemons along with their roles and responsibilities in the cluster.
• Should be able to troubleshoot issues in Cloudera services and fix those.
• Adding and removing nodes in the cluster.
• Rebalancing nodes in the cluster.
• Employ security using an authentication and authorization system such as Kerberos.
• Knowledge of changes required for migrating to Cloudera cloud version CDP
• Knowledge of Cloudera data services (CDW, CDE, CDF, CAI)
• Design Native Cloud Application Architectures and optimize applications for AWS
• Network connectivity, Direct Connect, VPN, VPC, Security group, NACL, Route 53
• Must have in depth AWS development experience (Containerization - Glue, Docker, Amazon EKS, Lambda, EC2, S3, Amazon DocumentDB, PostgreSQL)
• Strong knowledge of DevOps and CI/CD pipeline (GitHub, Jenkins)
• Scripting capability and the ability to develop AWS environments as code
• Hands-on AWS experience with at least 1 implementation (preferred in an Enterprise scale environment)
• Experience with core AWS platform architecture, including areas such as: Organizations, Account Design, VPC, Subnet, segmentation strategies.
• Environment and application automation
• CloudFormation and third-party automation approach/strategy
• AWS Cost Management and Optimization

Qualifications

• Extensive experience with Cloudera cloud data platform (CDP) and CDP Services and Big data knowledge.
• Proficiency in Terraform for infrastructure as code (IaC).
• Strong hands-on experience with Cloudera CDP and Hadoop ecosystem (Hive, Impala, HDFS, etc.)
• Experience with GitHub Actions or similar CI/CD tools (eg, Jenkins, GitLab CI).
• Solid Scripting skills in Shell and Python.
• Extensive experience in designing, provisioning, deploying and configurtion of the Cloudera cluster based on customer's need
• Extensive experience in AWS service - EC2, VPC, ELB, S3, RDS, Lambda, Route 53 etc and should be able to design and deploy cloudera cluster on AWS cloud
• Strong knowledge of data processing using Cloudera services
• Good understanding of CI/CD concepts, version control, and DevOps best practices.

Note: 3 days onsite required