Data Engineer

Datamatics Global Services Ltd

Tennessee

On-site

USD 122,000 - 182,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Datamatics Global Services Ltd in the UAE seeks a Data Engineer to design and operate automated data pipelines from source systems into lakehouse layers. You will build batch, streaming and CDC pipelines using Azure Data Factory and Fabric, while ensuring data quality, governance and secure access.

You will develop Silver/Gold transformations in SQL and PySpark, implement dimensional models, and publish metadata to Purview. Strong communication and documentation in English are required.

Qualifications

  • 4+ years in data engineering, preferably in regulated environments.
  • Advanced SQL, Python and PySpark/Spark skills.
  • Hands-on with Azure Data Factory, Microsoft Fabric, Databricks or Synapse.
  • Strong knowledge of Lakehouse architecture, Delta/Parquet, and SCDs.
  • Experience with CI/CD, source control and DataOps practices.
  • Familiarity with Purview metadata and data lineage, plus DQ tooling.
  • Awareness of UAE data regulation including UAE PDPL, Dubai Data Law and DESC ISR.
  • Excellent English communication and documentation skills.

Responsibilities

  • Build batch, incremental, CDC and streaming ingestion pipelines using Azure/Data Factory and Fabric pipelines.
  • Develop Silver- and Gold-layer transformations in SQL, PySpark and Spark notebooks.
  • Implement dimensional models, SCDs and surrogate keys as designed.
  • Integrate data quality checks and reconciliation with data quality engineer.
  • Publish metadata and lineage to Microsoft Purview from pipelines.
  • Implement scheduling, dependencies, parameterization and error handling.
  • Optimize pipelines for performance, partitioning, file formats (Delta/Parquet) and cost.
  • Apply access controls, masking and secure credential management per regulations.
  • Maintain code in source control and deploy via CI/CD with tests.
  • Support production operations, incident resolution and handover.

Skills

SQL
Python
PySpark
Azure Data Factory
Microsoft Fabric
Databricks
Synapse
Delta/Parquet
SCDs
Surrogate keys
CI/CD
Purview
DataOps

Tools

Azure Data Factory
Microsoft Fabric
Databricks
Synapse
GitHub
Azure DevOps
Microsoft Purview

Job description

Location: UAE-based (Travel required)

Duration: 6-month assignment

Summary (Data Engineer)

Timely, reliable data pipelines are what turn a platform design into business value. The Data Engineer builds and operates the pipelines that move data from source systems into the lakehouse and on to analytics and AI consumers, ensuring that the pipelines are automated, performant, secure and governed. The role works alongside the data architect, data quality engineer, business analysts and AI engineers, delivering production-grade pipelines on Azure / Microsoft Fabric, with metadata and lineage captured in Microsoft Purview.

Responsibilities
  • Build batch, incremental, CDC and streaming ingestion pipelines from source systems into the Bronze layer, using Azure Data Factory and Fabric pipelines.
  • Develop Silver- and Gold-layer transformations in SQL, PySpark and Spark notebooks, including cleansing, conformance, enrichment and business-rule logic.
  • Implement dimensional models, SCDs, surrogate keys and data marts according to the architect's designs.
  • Integrate DQ checks, validation, quarantine and reconciliation into pipelines in partnership with the Senior Data Quality Engineer.
  • Ensure that pipelines publish metadata and lineage to Microsoft Purview.
  • Implement scheduling, dependency management, parameterization, error handling and alerting.
  • Optimize pipelines, partitioning, file formats (Delta/Parquet) and compute usage for performance and cost efficiency.
  • Apply access controls, masking, encryption and secure credential management in line with UAE PDPL, the Dubai Data Law and DESC ISR.
  • Maintain code in source control and deploy through CI/CD pipelines, with automated tests and environment promotion.
  • Support production operations, incident resolution and runbooks, and coach counterparts so that the pipelines can be sustained after handover.
Must Have
  • 4+ years in data engineering, preferably within UAE government or regulated-sector environments.
  • Advanced SQL, Python and PySpark/Spark skills.
  • Hands-on experience with Azure Data Factory, Microsoft Fabric, Databricks or Synapse.
  • Strong understanding of Lakehouse/Medallion architecture, Delta/Parquet formats and dimensional modelling, including SCDs and surrogate keys.
  • Experience with CI/CD (Azure DevOps or GitHub), source control and DataOps practices.
  • Familiarity with Microsoft Purview metadata and lineage, and with DQ tooling.
  • Awareness of UAE data regulation, including UAE PDPL, the Dubai Data Law and DESC ISR.
  • Excellent English communication and documentation skills.
Good to Have
  • Arabic proficiency is an advantage.

4EZEuV9oVm

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

Talent Software Services, Inc. • Hartford (CT)

Remote
USD 90,000 - 130,000
Data Engineer
Data Engineer

AccelOne • United States

Remote
USD 49,000 - 98,000
Data Engineer
Data Engineer

Speridian • Austin (TX)

On-site
USD 140,000 - 190,000
Azure Data Engineer
Azure Data Engineer

Recru • Fort Worth (TX), Arlington (TX), Dallas (TX)

On-site
USD 120,000 - 150,000
Data Engineer
Data Engineer

TG Administration LLC • Scottsdale (AZ)

On-site
USD 95,000 - 130,000
Healthcare Benefits
Retirement Benefits
Time Off
Data Engineer
Data Engineer

ReturnPro LLP • Florida

On-site
USD 110,000 - 150,000
Azure Data Engineer
Azure Data Engineer

Zion Cloud Solutions • Washington

Remote
USD 120,000 - 180,000
Data Engineer
Data Engineer

Riseup Labs • United States

Remote
USD 120,000 - 180,000
General Leave
Festival Bonus
Annual Salary Review
Data Engineer
Data Engineer

Hirewell • Atlanta (GA)

On-site
USD 95,000 - 125,000
Data Engineer
Data Engineer

Pull Skil • United States

Remote
USD 83,000 - 152,000
Remote work