Senior Databricks Data Engineer - Onsite Abu Dhabi

Airswift; Abu Dhabi; On-site; AED 300,000 - 420,000; Full time

Job summary

Airswift is seeking a Senior Data Engineer for on-site work in Abu Dhabi, UAE. The role focuses on building and migrating data pipelines to Databricks, optimizing PySpark transformations, and aligning with CI/CD IaC practices within Azure-heavy environments.

The candidate should bring proven Databricks production experience, strong coding skills, and the ability to operate in pod-based delivery teams. Excellent English communication is required.

Qualifications

• Proven production-level Databricks experience.
• Strong hands-on coding capability.
• Experience building and optimizing modern data pipelines.
• Ability to work within pod-based delivery teams.
• Experience owning pipeline migration and rebuild activities from design through validation.
• Databricks Databricks hands-on experience.
• Delta Live Tables.
• Unity Catalog.
• Workflows.
• Production experience required (not lab-based experience).
• Data Engineering & Development PySpark.
• Advanced SQL.
• Data modelling.
• Schema design.
• Medallion architecture.
• Lakehouse architecture.
• DevOps & Deployment Modern CI/CD for data pipelines.
• Infrastructure as Code (IaC).
• Git-based deployment.
• Communication Strong written and verbal English skills.
• Ability to interact directly with Databricks Professional Services (DBX).

Responsibilities

• Implement Databricks pipelines using Delta Live Tables.
• Rewrite Foundry transforms into optimised PySpark on Databricks.
• Move from Foundry to modern CI/CD with IaC-managed Databricks workspaces.
• Tune Databricks Photon performance to match Foundry interactivity.
• Contribute to reconciliation engine between Palantir and Databricks outputs.
• Manage pipeline rebuild, validation, internal testing and UAT prep.

Skills

Databricks

PySpark

Delta Live Tables

Unity Catalog

CI/CD

IaC

SQL

Data modelling

GIT

Python

Airflow

Palantir Foundry

Code migration

Tools

Databricks

Delta Live Tables

Unity Catalog

dbt

Airflow

Terraform

Git

Infra as Code

Job description

Airswift is seeking a Senior Data Engineer for on-site work in Abu Dhabi, UAE. The role focuses on building and migrating data pipelines to Databricks, optimizing PySpark transformations, and aligning with CI/CD IaC practices within Azure-heavy environments.

The candidate should bring proven Databricks production experience, strong coding skills, and the ability to operate in pod-based delivery teams. Excellent English communication is required.