Databricks Data Quality Architect Lead | Hybrid UAE

iO Associates; Abu Dhabi; Hybrid; AED 350,000 - 550,000; Part time

Job summary

iO Associates is partnering with a leading UAE organisation to recruit a Databricks Data Quality & Remediation Architect / Lead for a hybrid role in Abu Dhabi. This senior, hands-on position combines Databricks architecture, data engineering and technical leadership on a major data platform delivery.

You will design scalable data quality capabilities, define Bronze-Silver-Gold layers, work with governance teams and lead a development team to deliver measurable quality KPIs and remediation

Qualifications

• 5+ years' experience in data engineering, Databricks or related roles.
• Ideally 2+ years in a Lead or Architect position.
• Strong hands on Databricks experience.
• Excellent Python and PySpark skills.

Responsibilities

• Design and implement a scalable Data Quality & Remediation Engine on Databricks.
• Define architecture across data profiling, quality rules, exception management, root-cause analysis and remediation.
• Build a metadata-driven data-quality framework.
• Define Bronze, Silver and Gold processing layers.
• Work with Data Governance teams and Data Stewards on Critical Data Elements and data-quality processes.
• Support Microsoft Purview integration, governance, lineage and auditability.
• Lead the development team and provide hands on technical direction.
• Own data-quality reporting, including quality KPIs, exceptions and remediation activity.

Skills

Databricks

Python

PySpark

Azure Databricks

Data quality

Leadership

Stakeholder management

Delta Lake

Databricks SQL

Tools

Azure Data Factory

ADLS Gen2

Azure DevOps

Purview

Job description

iO Associates is partnering with a leading UAE organisation to recruit a Databricks Data Quality & Remediation Architect / Lead for a hybrid role in Abu Dhabi. This senior, hands-on position combines Databricks architecture, data engineering and technical leadership on a major data platform delivery.

You will design scalable data quality capabilities, define Bronze-Silver-Gold layers, work with governance teams and lead a development team to deliver measurable quality KPIs and remediation