Data Engineer

Sutherland; Abu Dhabi; On-site; AED 250,000 - 420,000; Full time

Job summary

Sutherland in Abu Dhabi is seeking a Data Engineer to design and operate data pipelines from source systems into the lakehouse and onward to analytics and AI consumers.

You will work with Azure Data Factory, Fabric, Databricks or Synapse, implement governance with Purview, and optimize performance, cost, and security in production systems.

This role requires strong English communication and collaboration with architects, data quality engineers and BI teams.

Qualifications

• 4+ years in data engineering in UAE government or regulated sectors.
• Expertise in SQL, Python and PySpark/Spark.
• Experience with Azure Data Factory, Fabric, Databricks or Synapse.
• Knowledge of Purview metadata and lineage.
• English communication strong; Arabic is a plus.

Responsibilities

• Pipeline development: Build batch, incremental, CDC and streaming ingestion pipelines from source systems into the Bronze layer using Azure Data Factory and Fabric pipelines.
• Transformation: Develop Silver- and Gold-layer transformations in SQL, PySpark and Spark notebooks.
• Data modelling: Implement dimensional models, SCDs, surrogate keys and data marts.
• Quality controls: Integrate DQ checks and validation into pipelines.
• Metadata and lineage: Publish metadata and lineage to Microsoft Purview.
• Orchestration: Implement scheduling, dependency management and error handling.

Skills

Advanced SQL

Python

PySpark

Spark

CI/CD practices

English communication

Tools

Azure Data Factory

Microsoft Fabric

Databricks

Synapse

Azure DevOps

GitHub

Purview

Job description

Timely, reliable data pipelines are what turn a platform design into business value. The Data Engineer builds and operates pipelines that move data from source systems into the lakehouse and on to analytics and AI consumers. They ensure that the pipelines are automated, performant, secure and governed.

The role works alongside the data architect, data quality engineer, business analysts and AI engineers. Its job is to deliver production-grade pipelines on Azure / Microsoft Fabric, with metadata and lineage captured in Microsoft Purview.

Key Responsibilities on the Client's Engagement

• Pipeline development: Build batch, incremental, CDC and streaming ingestion pipelines from source systems into the Bronze layer, using Azure Data Factory and Fabric pipelines.
• Transformation: Develop Silver- and Gold-layer transformations in SQL, PySpark and Spark notebooks, including cleansing, conformance, enrichment and business-rule logic.
• Data modelling implementation: Implement dimensional models, SCDs, surrogate keys and data marts according to the architect's designs.
• Embedded quality controls: Integrate DQ checks, validation, quarantine and reconciliation into pipelines in partnership with the Senior Data Quality Engineer.
• Metadata and lineage: Ensure that pipelines publish metadata and lineage to Microsoft Purview.
• Orchestration and automation: Implement scheduling, dependency management, parameterization, error handling and alerting.
• Performance and cost: Optimize pipelines, partitioning, file formats (Delta/Parquet) and compute usage for performance and cost efficiency.
• Security: Apply access controls, masking, encryption and secure credential management in line with UAE PDPL, the Dubai Data Law and DESC ISR.
• DevOps / DataOps: Maintain code in source control and deploy through CI/CD pipelines, with automated tests and environment promotion.
• Operations and knowledge transfer: Support production operations, incident resolution and runbooks, and coach Client's engineers so that the pipelines can be sustained after handover.

Qualifications and Experience

• 4+ years in data engineering, preferably within UAE government or regulated-sector environments.
• Advanced SQL, Python and PySpark/Spark skills.
• Hands-on experience with Azure Data Factory, Microsoft Fabric, Databricks or Synapse.
• Strong understanding of Lakehouse/Medallion architecture, Delta/Parquet formats and dimensional modelling, including SCDs and surrogate keys.
• Experience with CI/CD (Azure DevOps or GitHub), source control and DataOps practices.
• Familiarity with Microsoft Purview metadata and lineage, and with DQ tooling.
• Awareness of UAE data regulation, including UAE PDPL, the Dubai Data Law and DESC ISR.
• Excellent English communication and documentation skills; Arabic proficiency is an advantage.