Senior Cloudera ETL Developer

Indsafri India Private Limited

South Africa

Remote

ZAR 1.200.000 - 1.900.000

Vollzeit

Vor 13 Tagen

Erhalte eine Antwort von diesem Arbeitgeber

Dein Lebenslauf und dein Anschreiben, genau auf das zugeschnitten, was gesucht wird.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

Indsafri India Private Limited is seeking a Senior Cloudera ETL Developer with 7+ years of experience to design, build, and optimize robust data pipelines. You will work across NiFi, Spark, and SQL to deliver scalable ETL solutions in a Cloudera environment.

The role emphasizes data governance, quality, and security, with collaboration across data engineers, scientists, and business analysts to translate requirements into performant data platforms.

Qualifikationen

  • 7+ years of ETL development and data engineering experience.
  • Proven expertise with Apache NiFi for designing data flows.
  • Strong Spark experience (PySpark/Scala) for large-scale processing.
  • Advanced SQL skills and query optimization.
  • Experience with Cloudera ecosystem (HDFS, Hive, Impala).
  • Knowledge of data warehousing, data modeling and DB design.

Aufgaben

  • Design, develop, and maintain high-performance ETL pipelines using Cloudera tech (NiFi, Spark).
  • Write efficient SQL to extract, transform, and load data into warehouses and lakes.
  • Develop and optimize Spark jobs for large-scale processing.
  • Build data flows with NiFi and orchestrate ETL processes.
  • Collaborate with data engineers, scientists, and analysts to meet data needs.
  • Troubleshoot pipeline issues and improve performance and data quality.
  • Implement data governance, quality, and security in ETL processes.
  • Document ETL processes, data models, and specs.
  • Mentor junior developers and grow the team’s capabilities.
  • Stay updated on big data trends and ETL tech.

Kenntnisse

ETL development
Data engineering
SQL proficiency
Python/Bash scripting
Big data concepts

Tools

Apache NiFi
Apache Spark
Hadoop HDFS
Hive/Impala
CI/CD pipelines
Git

Jobbeschreibung

We are seeking a highly experienced Senior Cloudera ETL Developer with a minimum of 7 years of experience in designing, developing, and optimizing robust data pipelines. The ideal candidate will have a strong background in the Cloudera ecosystem, including extensive hands‑on experience with NiFi, Spark, and SQL for building and managing complex ETL processes. This role requires a proactive individual who can translate business requirements into efficient and scalable data solutions.

Key Responsibilities
  • Design, develop, and maintain high-performance ETL pipelines using Cloudera technologies, primarily Apache NiFi and Apache Spark.
  • Write efficient and complex SQL queries to extract, transform, and load data from various sources into data warehouses and data lakes.
  • Develop and optimize Spark jobs for large-scale data processing, ensuring performance and cost-efficiency.
  • Utilize Apache NiFi to build data flows, manage data ingestion, and orchestrate complex ETL processes.
  • Collaborate with data engineers, data scientists, and business analysts to understand data requirements and deliver effective solutions.
  • Troubleshoot and resolve issues related to data pipelines, performance bottlenecks, and data quality.
  • Implement data governance, data quality, and data security best practices within ETL processes.
  • Document ETL processes, data models, and technical specifications.
  • Mentor junior developers and contribute to the overall technical growth of the team.
  • Stay updated with the latest trends and technologies in big data and ETL development.
Required Skills and Qualifications
  • 7+ years of professional experience in ETL development and data engineering.
  • Proven expertise in Apache NiFi for designing and implementing data flows.
  • Strong proficiency in Apache Spark (PySpark or Scala) for data processing and transformation.
  • Advanced SQL skills with experience in complex query writing and optimization.
  • Extensive experience with the Cloudera ecosystem (e.g., HDFS, Hive, Impala).
  • Solid understanding of data warehousing concepts, data modeling, and database design.
  • Experience with scripting languages such as Python or Bash.
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration abilities.
Preferred Qualifications
  • Experience with other big data technologies (e.g., Kafka, HDFS).
  • Familiarity with cloud-based data platforms (e.g., AWS, Azure, GCP).
  • Experience with CI/CD pipelines and version control systems (e.g., Git).
  • Knowledge of data visualization tools.
Skills

GIT Python AWS GCP Azure analytical skills SQL PySpark CI/CD pipelines communication Scala Problem Solving Bash Collaboration Data Visualization Data Warehousing Data Modeling apache spark hdfs database design kafka impala apache nifi Hive Data Engineering ETL Development Cloudera ecosystem

Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

Senior Cloudera ETL Architect: Spark, NiFi & Data Pipelines
Senior Cloudera ETL Architect: Spark, NiFi & Data Pipelines

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.900.000
Senior Cloudera Spark Developer
Senior Cloudera Spark Developer

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.800.000
Senior Cloudera Data Engineer
Senior Cloudera Data Engineer

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.800.000
Senior Cloudera Hadoop Developer
Senior Cloudera Hadoop Developer

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 2.200.000
Senior Cloudera Platform Engineer
Senior Cloudera Platform Engineer

Indsafri India Private Limited • Südafrika

Remote
ZAR 900.000 - 1.500.000
Senior Data Engineer: Spark, Databricks & Cloud Pipelines
Senior Data Engineer: Spark, Databricks & Cloud Pipelines

Boardroom Appointments • Sandton

Vor Ort
ZAR 900.000 - 1.500.000
Senior Cloudera Data Engineer — Scalable Pipeline Lead
Senior Cloudera Data Engineer — Scalable Pipeline Lead

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.800.000
Intermediate Cloudera Platform Engineer
Intermediate Cloudera Platform Engineer

Indsafri India Private Limited • Südafrika

Remote
ZAR 600.000 - 1.000.000
Senior Data Engineer - Cloud Data Architect & Analytics
Senior Data Engineer - Cloud Data Architect & Analytics

Boardroom Appointments • Johannesburg

Vor Ort
ZAR 800.000 - 1.200.000
Senior Data Engineer
Senior Data Engineer

Skywaves Rise • Südafrika

Remote
ZAR 900.000 - 1.300.000
Competitive salary
Comprehensive benefits
Hybrid/remote work options