Dubai-based PySpark Data Engineer | CDP & ETL Expert

HireAlpha; Dubai; On-site; AED 320,000 - 520,000; Full time

Job summary

HireAlpha is seeking an experienced Data Engineer (PySpark) to design and maintain scalable ETL pipelines on the Cloudera Data Platform in Dubai. You will transform raw data into trusted datasets for analytics, dashboards, and data science.

You will optimize PySpark jobs, ensure data quality, and collaborate with analysts and product teams to deliver reliable data assets. Strong scripting, Linux, and orchestration experience with Oozie or Airflow are required.

Qualifications

• 5+ years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform.
• Design, develop, and maintain scalable ETL pipelines using PySpark on CDP.
• Implement data ingestion from multiple sources to data lake or data warehouse on CDP.
• Ensure data quality, validation, and monitoring throughout pipelines.
• Automate data workflows with Oozie, Airflow, or similar tools.
• Collaborate with data teams and document processes.

Responsibilities

• 5+ years in data engineering with PySpark and CDP.
• Design, build and maintain scalable ETL pipelines.
• Ingest data from various sources to data lake/warehouse on CDP.
• Optimize PySpark performance and resource usage.
• Implement data quality checks and monitoring.
• Automate workflows with Oozie or Airflow.
• Monitor pipelines and troubleshoot issues.
• Document data processes and pipelines.
• Collaborate with stakeholders across teams.

Skills

PySpark

ETL development

Data pipelines

Linux scripting

Collaboration

Tools

Cloudera Data Platform

Hive

Impala

HDFS

Oozie

Airflow

Kafka

Job description

HireAlpha is seeking an experienced Data Engineer (PySpark) to design and maintain scalable ETL pipelines on the Cloudera Data Platform in Dubai. You will transform raw data into trusted datasets for analytics, dashboards, and data science.

You will optimize PySpark jobs, ensure data quality, and collaborate with analysts and product teams to deliver reliable data assets. Strong scripting, Linux, and orchestration experience with Oozie or Airflow are required.