We are seeking a highly experienced and motivated Senior Cloudera Platform Engineer to join our dynamic data engineering team. The ideal candidate will have a strong background in designing, implementing, and managing large-scale Cloudera Data Platform (CDP) environments. This role requires deep expertise in distributed systems, cloud technologies, Kubernetes, Linux administration, and automation to ensure the stability, performance, and scalability of our data infrastructure.
Key Responsibilities
- Design, deploy, and maintain Cloudera Data Platform (CDP) environments across various cloud providers (AWS, Azure, GCP) and on-premises infrastructure.
- Manage and optimize Hadoop ecosystem components within CDP, including HDFS, YARN, Hive, Impala, Spark, Kafka, and HBase.
- Administer and orchestrate Kubernetes clusters integrated with CDP for containerized data services.
- Perform regular system monitoring, performance tuning, and capacity planning for the Cloudera platform.
- Develop and implement automation scripts for deployment, configuration, patching, and maintenance of the CDP environment using tools like Ansible, Python, or Shell scripting.
- Troubleshoot and resolve complex technical issues related to the Cloudera platform and its integrated services.
- Ensure data security, compliance, and governance best practices are implemented and maintained.
- Collaborate with data scientists, data engineers, and application developers to support their data needs and provide platform expertise.
- Stay up to date with the latest trends and technologies in big data, cloud computing, and containerization.
- Document platform architecture, configurations, and operational procedures.
Required Skills and Qualifications
- 7+ years of hands-on experience in designing, deploying, and managing Cloudera Data Platform (CDP) or other Hadoop distributions.
- In-depth knowledge of cloud platforms (AWS, Azure, or GCP) and their relevant services for data platforms.
- Strong experience with containerization technologies, particularly Kubernetes, including deployment, management, and troubleshooting.
- Proficiency in scripting languages such as Python, Bash, or PowerShell for automation.
- Familiarity with CI/CD principles and tools.
- Excellent understanding of networking concepts and security best practices.
- Strong problem-solving and analytical skills with the ability to troubleshoot complex distributed systems.
- Effective communication and collaboration skills.
Preferred Qualifications
- Cloudera certifications (e.g., CCA Administrator, CCA Data Scientist).
- Experience with cloud-native data services and their integration with CDP.
- Knowledge of monitoring tools like Prometheus, Grafana, or Splunk.
- Experience with Infrastructure as Code (IaC) tools such as Terraform or CloudFormation.
- Familiarity with data governance and data lineage tools.
Skills
Python AWS GCP Azure analytical skills Problem-Solving communication Kubernetes Grafana Prometheus CI/CD Bash Powershell Collaboration Data Governance CloudFormation Infrastructure as Code (IaC) Ubuntu Ansible Terraform hdfs shell scripting centos splunk networking rhel kafka impala spark Hive Security YARN HBase Hadoop Ecosystem Linux system administration Cloudera Data Platform (CDP) Cloudera certifications CCA Administrator CCA Data Scientist cloud-native data services Data lineage tools