Senior Cloudera Data Engineer — Scalable Pipeline Lead

Indsafri India Private Limited

South Africa

Remote

ZAR 1.200.000 - 1.800.000

Vollzeit

Vor 13 Tagen

Eine vollständige Bewerbung in einer Minute

Maßgeschneiderter Lebenslauf und Anschreiben – versandbereit.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

Indsafri India Private Limited seeks a highly experienced Senior Cloudera Data Engineer to design, develop, and optimize complex data pipelines within the Cloudera ecosystem. You will translate business requirements into robust data engineering solutions and collaborate with data scientists and stakeholders.

The role requires deep expertise in CDP components (HDFS, YARN, Spark, Hive, Impala, Kudu) and strong Python (PySpark) and SQL skills.

Qualifikationen

  • 7+ years of hands-on experience in data engineering with big data technologies.
  • Proven expertise in Cloudera Data Platform (CDP) and components like HDFS, YARN, Spark, Hive, Impala, Kudu.
  • Strong programming skills in Python (PySpark) and SQL; experience in designing, developing, and optimizing ETL/ELT workflows.
  • Solid understanding of data warehousing concepts and data modeling techniques.
  • Experience with distributed computing frameworks like Apache Spark.
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration abilities.

Aufgaben

  • Design, build, and maintain scalable and reliable data pipelines using Cloudera CDP (CDH/HDP).
  • Develop and optimize ETL/ELT processes using Spark, PySpark, Hive, and Impala.
  • Write efficient and complex SQL queries for data extraction, transformation, and analysis.
  • Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver impactful solutions.
  • Implement data quality checks and ensure data integrity across various data platforms.
  • Monitor and troubleshoot data pipeline performance, identifying and resolving issues proactively.
  • Contribute to architectural decisions and best practices for big data solutions.
  • Mentor junior data engineers and share expertise within the team.
  • Stay up to date with the latest trends and technologies in big data and cloud data engineering.

Kenntnisse

Cloudera CDP
PySpark
SQL
ETL/ELT
Big data
Python
Spark
Data Warehousing
Data Modeling
Data Governance
Security best practices

Tools

Docker
Kubernetes
Kafka
Kinesis

Jobbeschreibung

Indsafri India Private Limited seeks a highly experienced Senior Cloudera Data Engineer to design, develop, and optimize complex data pipelines within the Cloudera ecosystem. You will translate business requirements into robust data engineering solutions and collaborate with data scientists and stakeholders.

The role requires deep expertise in CDP components (HDFS, YARN, Spark, Hive, Impala, Kudu) and strong Python (PySpark) and SQL skills.

Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

Senior Spark Engineer - Cloudera Data Platform
Senior Spark Engineer - Cloudera Data Platform

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.800.000
Senior Cloudera Data Engineer
Senior Cloudera Data Engineer

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.800.000
Senior Cloudera Hadoop Architect for Big Data Solutions
Senior Cloudera Hadoop Architect for Big Data Solutions

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 2.200.000
Senior Cloudera ETL Architect: Spark, NiFi & Data Pipelines
Senior Cloudera ETL Architect: Spark, NiFi & Data Pipelines

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.900.000
Senior Cloudera CDP Platform Engineer | Scale &Automation
Senior Cloudera CDP Platform Engineer | Scale &Automation

Indsafri India Private Limited • Südafrika

Remote
ZAR 900.000 - 1.500.000
Senior Cloudera Spark Developer
Senior Cloudera Spark Developer

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.800.000
Senior Cloudera Data Architect — CDP & Data Governance
Senior Cloudera Data Architect — CDP & Data Governance

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.800.000
Senior Cloudera Platform Engineer
Senior Cloudera Platform Engineer

Indsafri India Private Limited • Südafrika

Remote
ZAR 900.000 - 1.500.000
Senior Cloudera Hadoop Developer
Senior Cloudera Hadoop Developer

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 2.200.000
CDP Platform Engineer — Kubernetes & Cloud Automation
CDP Platform Engineer — Kubernetes & Cloud Automation

Indsafri India Private Limited • Südafrika

Remote
ZAR 600.000 - 1.000.000