Data Engineer

Datamatics Technologies

Abu Dhabi

Sur place

AED 240 000 - 360 000

Plein temps

Il y a 20 heures
Soyez parmi les premiers à postuler

Obtenez une réponse de cet employeur

Votre CV et votre lettre de motivation, adaptés exactement à ce que l’employeur recherche.

Passez les filtres ATS

Résumé du poste

Datamatics Technologies in Abu Dhabi seeks a Data Engineer to design, build and operate data pipelines that move data from source systems into the lakehouse and onward to analytics and AI consumers, ensuring automation, performance, security and governance.

You will work with Azure Data Factory and Fabric pipelines, develop SQL and PySpark transformations, implement dimensional models and SCDs, and ensure metadata and lineage are captured in Microsoft Purview, with CI/CD and robust runbooks.

Qualifications

  • 4+ years in data engineering, preferably in regulated environments in the UAE.
  • Expertise in SQL, Python and PySpark/Spark.
  • Hands-on with Azure Data Factory and modern lakehouse tools (Databricks/Synapse).
  • Strong knowledge of data governance, Purview metadata and lineage.
  • Experience with CI/CD and DataOps practices.
  • Awareness of UAE data regulations (PDPL, Dubai Data Law).

Responsabilités

  • Build batch, incremental, CDC and streaming ingestion pipelines from source systems into the Bronze layer.
  • Develop Silver- and Gold-layer transformations in SQL and PySpark with cleansing and conformance.
  • Implement dimensional models, SCDs and data marts per architectural designs.
  • Integrate data quality checks and reconciliation into pipelines, partnering with QA engineers.
  • Publish metadata and lineage to Microsoft Purview; ensure governance coverage.
  • Ensure CI/CD, deployment automation and test coverage; support handover and runbooks.

Connaissances

Advanced SQL
Python
PySpark / Spark
Azure Data Factory
Microsoft Fabric
Databricks / Synapse
Lakehouse / Medallion
Dimensional modeling / SCDs
CI/CD / DataOps
Purview metadata & lineage
English communication

Outils

Azure Data Factory
Microsoft Fabric
Databricks
Synapse

Description du poste

Timely, reliable data pipelines are what turn a platform design into business value. The Data Engineer builds and operates the pipelines that move data from source systems into the lakehouse and on to analytics and AI consumers, ensuring that the pipelines are automated, performant, secure and governed. The role works alongside the data architect, data quality engineer, business analysts and AI engineers, delivering production‑grade pipelines on Azure / Microsoft Fabric, with metadata and lineage captured in Microsoft Purview.

Responsibilities
  • Build batch, incremental, CDC and streaming ingestion pipelines from source systems into the Bronze layer, using Azure Data Factory and Fabric pipelines
  • Develop Silver- and Gold-layer transformations in SQL, PySpark and Spark notebooks, including cleansing, conformance, enrichment and business-rule logic
  • Implement dimensional models, SCDs, surrogate keys and data marts according to the architect's designs
  • Integrate DQ checks, validation, quarantine and reconciliation into pipelines in partnership with the Senior Data Quality Engineer
  • Ensure that pipelines publish metadata and lineage to Microsoft Purview
  • Implement scheduling, dependency management, parameterization, error handling and alerting
  • Optimize pipelines, partitioning, file formats (Delta/Parquet) and compute usage for performance and cost efficiency
  • Apply access controls, masking, encryption and secure credential management in line with UAE PDPL, the Dubai Data Law and DESC ISR
  • Maintain code in source control and deploy through CI/CD pipelines, with automated tests and environment promotion
  • Support production operations, incident resolution and runbooks, and coach counterparts so that the pipelines can be sustained after handover
Must Have
  • 4+ years in data engineering, preferably within UAE government or regulated-sector environments
  • Advanced SQL, Python and PySpark/Spark skills
  • Hands-on experience with Azure Data Factory, Microsoft Fabric, Databricks or Synapse
  • Strong understanding of Lakehouse/Medallion architecture, Delta/Parquet formats and dimensional modelling, including SCDs and surrogate keys
  • Experience with CI/CD (Azure DevOps or GitHub), source control and DataOps practices
  • Familiarity with Microsoft Purview metadata and lineage, and with DQ tooling
  • Awareness of UAE data regulation, including UAE PDPL, the Dubai Data Law and DESC ISR
  • Excellent English communication and documentation skills
Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

Data Engineer
Data Engineer

Datamatics Technologies • Sharjah

Sur place
AED 240 000 - 360 000
Data Engineer
Data Engineer

Datamatics Global Services Ltd • Dubai

Sur place
AED 240 000 - 360 000
Data Engineer
Data Engineer

Datamatics Technologies LLC • Abu Dhabi

Sur place
AED 240 000 - 420 000
Data Engineer
Data Engineer

Datamatics Technologies LLC • Dubai

Sur place
AED 190 000 - 290 000
Data Engineer
Data Engineer

Datamatics Global Services Ltd • Abu Dhabi

Sur place
AED 201 000 - 357 000
Data Engineer
Data Engineer

Datamatics Global Services Ltd • Sharjah

Sur place
AED 180 000 - 360 000
Data Engineer
Data Engineer

Datamatics Technologies LLC • Sharjah

Sur place
AED 134 000 - 201 000
Data Quality Engineer
Data Quality Engineer

Datamatics Technologies • Abu Dhabi

Sur place
AED 300 000 - 460 000
Data Engineer - Azure Fabric & Lakehouse (UAE, 6m)
Data Engineer - Azure Fabric & Lakehouse (UAE, 6m)

Datamatics Global Services Ltd • Sharjah

Sur place
AED 180 000 - 360 000
Azure Data Engineer (6-Month) - UAE, Data Lake & Pipelines
Azure Data Engineer (6-Month) - UAE, Data Lake & Pipelines

Datamatics Technologies LLC • Sharjah

Sur place
AED 134 000 - 201 000
Lieux principaux