Azure Databricks Data Engineer - PySpark Expert

Duncan & Ross

Dubai

On-site

AED 240,000 - 360,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Duncan & Ross is hiring a Data Engineer for a leading client in Dubai. You will design, build and run scalable data pipelines and data warehouse solutions on Azure and Databricks, turning raw data into trusted analytics-ready data.

3-5+ years of hands-on data engineering experience is required. You will work with Python/PySpark, SQL, and cloud services to deliver robust data products, while ensuring data quality and governance across heterogeneous sources.

Qualifications

  • 3-5+ years hands-on data engineering experience.
  • Data pipelines: ETL/ELT pipelines for batch/incremental loads.
  • SQL & relational databases: advanced SQL with SQL Server/Azure SQL/PostgreSQL/Oracle.
  • Big data: hands-on Apache Spark, performance tuning for large datasets.
  • Cloud: strong Azure (Data Lake Storage, Data Factory, Synapse) & Databricks experience.
  • Data warehousing/lakehouse design on Azure & Databricks (dimensional modelling; Delta Lake).
  • Programming: strong Python and PySpark skills.
  • Orchestration: Azure Data Factory, Databricks Workflows, Airflow or similar.
  • Data quality, lineage, metadata, access controls; governance.

Responsibilities

  • Design, build and maintain ETL/ELT pipelines ingesting data from multiple sources into Azure and Databricks.
  • Develop data transformations in Python and PySpark, optimising Spark jobs for performance and cost.
  • Write efficient SQL for data modelling and analysis across databases.
  • Build and manage data warehouses and lakehouses on Azure and Databricks, including Delta Lake architectures.
  • Orchestrate data workflows with ADF, Databricks Workflows or Airflow, with monitoring and alerts.
  • Implement data quality checks, validation rules, and reconciliation processes.
  • Apply governance with cataloguing, lineage, access control and security (Unity Catalog / Purview).
  • Collaborate with data analysts/scientists and stakeholders to deliver data products.
  • Troubleshoot failures and performance issues; continuously improve reliability.
  • Document pipelines, models and processes; follow CI/CD and version control.

Skills

Data pipelines
SQL & databases
PySpark
Azure Databricks
Python
Azure Data Factory
Spark tuning
ETL/ELT
Git CI/CD
Agile
English proficiency

Education

Bachelor's degree

Tools

Databricks
Spark
Azure Data Factory
Airflow
CI/CD (Azure DevOps/GitHub Actions)
Power BI
Delta Lake
Purview
Terraform

Job description

Duncan & Ross is hiring a Data Engineer for a leading client in Dubai. You will design, build and run scalable data pipelines and data warehouse solutions on Azure and Databricks, turning raw data into trusted analytics-ready data.

3-5+ years of hands-on data engineering experience is required. You will work with Python/PySpark, SQL, and cloud services to deliver robust data products, while ensuring data quality and governance across heterogeneous sources.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Azure Databricks Data Engineer (PySpark)
Azure Databricks Data Engineer (PySpark)

Duncan & Ross Consulting • Dubai

On-site
AED 250,000 - 380,000
DATA ENGINEER - AZURE DATABRICKS & PYSPARK
DATA ENGINEER - AZURE DATABRICKS & PYSPARK

Duncan & Ross Consulting • Dubai

On-site
AED 250,000 - 380,000
DATA ENGINEER - AZURE DATABRICKS & PYSPARK
DATA ENGINEER - AZURE DATABRICKS & PYSPARK

Duncan & Ross • Dubai

On-site
AED 240,000 - 360,000
Data Engineer, Azure & Databricks + Airflow Expert
Data Engineer, Azure & Databricks + Airflow Expert

Tech Junction Ltd • Dubai

On-site
AED 240,000 - 420,000
Cloud Data Engineer — Databricks, Azure & Kafka
Cloud Data Engineer — Databricks, Azure & Kafka

Dicetek • Abu Dhabi

On-site
AED 260,000 - 380,000
Dubai-based Data Engineer | Azure, Databricks & Pipelines
Dubai-based Data Engineer | Azure, Databricks & Pipelines

techcarrot FZ • Dubai

On-site
AED 220,000 - 420,000
Senior Data Engineer: Pipelines, Spark & Azure DevOps
Senior Data Engineer: Pipelines, Spark & Azure DevOps

techcarrot FZ • Dubai

On-site
AED 240,000 - 480,000
Dubai-based PySpark Data Engineer | CDP & ETL Expert
Dubai-based PySpark Data Engineer | CDP & ETL Expert

HireAlpha • Dubai

On-site
AED 320,000 - 520,000
Azure Data Engineer: Big Data, Spark, Synapse & DevOps
Azure Data Engineer: Big Data, Spark, Synapse & DevOps

IBS Software • United Arab Emirates

On-site
AED 300,000 - 540,000
Data Engineer (PySpark)
Data Engineer (PySpark)

Black Pearl Consult • Abu Dhabi

On-site
AED 260,000 - 380,000
Top locations