Senior Spark Engineer - Cloudera Data Platform

Indsafri India Private Limited

South Africa

Remote

ZAR 1.200.000 - 1.800.000

Vollzeit

Vor 13 Tagen

Erhalte eine Antwort von diesem Arbeitgeber

Dein Lebenslauf und dein Anschreiben, genau auf das zugeschnitten, was gesucht wird.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

Indsafri India Private Limited in South Africa seeks a Senior Cloudera Spark Developer to design, develop, and optimize large-scale data pipelines using Spark, PySpark, Scala, and SQL within a Cloudera environment.

You will mentor juniors, review code, and collaborate with data engineers and scientists to deliver robust data solutions on HDFS, Hive/Impala, and Spark on YARN, while improving performance and reducing costs.

Qualifikationen

  • 7+ years of experience in big data development, with a significant focus on Apache Spark.
  • Proficiency in PySpark and Scala for developing Spark applications.
  • Expertise in SQL for data manipulation and complex querying.

Aufgaben

  • Design, develop, and implement high-performance Spark data processing applications.
  • Optimize Spark jobs on Cloudera for performance and cost.
  • Develop and maintain complex SQL queries for data extraction and analysis.
  • Collaborate with data engineers, data scientists, and stakeholders to translate requirements.
  • Build and manage data pipelines for batch and real-time processing.
  • Perform code reviews, testing, and debugging of Spark applications.
  • Troubleshoot performance bottlenecks in Spark environments.
  • Mentor junior developers and share best practices.

Kenntnisse

Spark
PySpark
Scala
SQL
Cloudera
HDFS
Hive
Impala
YARN
ETL/ELT

Tools

Kafka
Flink
NoSQL
Airflow
CI/CD

Jobbeschreibung

Indsafri India Private Limited in South Africa seeks a Senior Cloudera Spark Developer to design, develop, and optimize large-scale data pipelines using Spark, PySpark, Scala, and SQL within a Cloudera environment.

You will mentor juniors, review code, and collaborate with data engineers and scientists to deliver robust data solutions on HDFS, Hive/Impala, and Spark on YARN, while improving performance and reducing costs.

Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

Senior Cloudera Hadoop Architect for Big Data Solutions
Senior Cloudera Hadoop Architect for Big Data Solutions

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 2.200.000
Senior Cloudera Data Engineer — Scalable Pipeline Lead
Senior Cloudera Data Engineer — Scalable Pipeline Lead

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.800.000
Senior Cloudera Spark Developer
Senior Cloudera Spark Developer

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.800.000
Senior Cloudera ETL Architect: Spark, NiFi & Data Pipelines
Senior Cloudera ETL Architect: Spark, NiFi & Data Pipelines

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.900.000
Senior Cloudera Data Engineer
Senior Cloudera Data Engineer

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 1.800.000
Senior Cloudera CDP Platform Engineer | Scale &Automation
Senior Cloudera CDP Platform Engineer | Scale &Automation

Indsafri India Private Limited • Südafrika

Remote
ZAR 900.000 - 1.500.000
Senior Cloudera Hadoop Developer
Senior Cloudera Hadoop Developer

Indsafri India Private Limited • Südafrika

Remote
ZAR 1.200.000 - 2.200.000
Senior Cloudera Platform Engineer
Senior Cloudera Platform Engineer

Indsafri India Private Limited • Südafrika

Remote
ZAR 900.000 - 1.500.000
Senior Big Data Engineer — Scalable Pipelines (Hybrid)
Senior Big Data Engineer — Scalable Pipelines (Hybrid)

Placements24 • Pretoria

Hybrid
ZAR 900.000 - 1.500.000
Hybrid work model
Professional development opportunities
Conferences and training
Senior Data Engineer: Spark, Databricks & Cloud Pipelines
Senior Data Engineer: Spark, Databricks & Cloud Pipelines

Boardroom Appointments • Sandton

Vor Ort
ZAR 900.000 - 1.500.000