Senior Data Engineer

Dautom; Abu Dhabi; On-site; AED 320,000 - 460,000; Full time

Job summary

Dautom is hiring a Senior Data Engineer in Abu Dhabi to join a client project. You will design and manage enterprise data products, build scalable data pipelines, and optimize platforms across batch, real-time, and streaming domains.

The role requires deep expertise in CDP, Lakehouse architectures, and data governance. Experience with Trino, Delta Lake/Delta Sharing, and data mesh concepts is highly valued.

Qualifications

• Minimum 10 years in Data Engineering Domain.
• Experience in UAE or Middle East is desirable.
• Education : B.Tech/BE
• Strong hands-on experience with Cloudera Data Platform (CDP) and modern Data Lakehouse architectures.
• Expertise in SQL, Python, and data pipeline development for large-scale data integration and transformation.
• Experience with Talend, Apache Spark, Kafka, and Airflow for batch, real-time, and streaming data processing.
• Strong knowledge of Trino, Delta Lake/Iceberg, and Delta Sharing for data virtualization, storage, and sharing.
• Experience in data modelling, including Dimensional, Data Vault, and Lakehouse modelling techniques, Medallion Architecture and Data Mesh principles.
• Hands-on experience with OpenMetadata, Apache Ranger, metadata management, data lineage, and data governance frameworks.
• Experience in API integration, data services, and enterprise data product development.
• Proficiency in Git, CI/CD, DevOps practices, and performance optimization

Responsibilities

• Design, build, and manage enterprise data products and datasets in accordance with data architecture, governance, security, and data-sharing standards.
• Develop and maintain scalable data ingestion, integration, and transformation pipelines using batch, real-time, and streaming processing patterns.
• Engineer and optimize data solutions across the enterprise data platform, ensuring performance, reliability, scalability, and reusability.
• Design and maintain logical and physical data models to support analytics, AI, and business consumption.
• Implement data quality, validation, monitoring, and observability frameworks to ensure trusted and reliable data assets.
• Administer and support platform services, including orchestration, metadata management, data cataloging, lineage, and access controls.
• Develop APIs, data virtualization, and data-sharing services to enable secure and efficient access to enterprise data.
• Monitor, troubleshoot, and optimize platform operations, data pipelines, and resource utilization to ensure service availability and operational excellence.
• Maintain technical documentation, standards, and operational runbooks to support governance, compliance, and knowledge transfer.

Skills

Data Engineering

SQL

Python

Data Pipeline

CDP

Spark

Kafka

Airflow

Trino

Delta Lake/ Iceberg

Data Mesh

OpenMetadata

Apache Ranger

Git/CI-CD/DevOps

Education

B.Tech/BE

Tools

Talend

Apache Spark

Kafka

Airflow

Trino

Delta Lake/ Iceberg

OpenMetadata

Apache Ranger

Job description

In this role, you will collaborate closely with one of our esteemed clients—a globally recognized leader in their industry, distinguished by their commitment to excellence, innovation, and delivering exceptional value. As a trusted IT consulting partner, Dautom is supporting their strategic initiatives by connecting them with exceptional talent to drive business growth and transformation.

About the role :

Role Name : Senior Data Engineer

Job Location : Abu Dhabi, UAE

Required Skills & Experience :

• Minimum 10 years in Data Engineering Domain
• Experience in UAE or Middle East is desirable
• Education : B.Tech/BE
• Strong hands-on experience with Cloudera Data Platform (CDP) and modern Data Lakehouse architectures.
• Expertise in SQL, Python, and data pipeline development for large-scale data integration and transformation.
• Experience with Talend, Apache Spark, Kafka, and Airflow for batch, real-time, and streaming data processing.
• Strong knowledge of Trino, Delta Lake/Iceberg, and Delta Sharing for data virtualization, storage, and sharing.
• Experience in data modelling, including Dimensional, Data Vault, and Lakehouse modelling techniques, Medallion Architecture and Data Mesh principles.
• Hands-on experience with OpenMetadata, Apache Ranger, metadata management, data lineage, and data governance frameworks.
• Experience in API integration, data services, and enterprise data product development.
• Proficiency in Git, CI/CD, DevOps practices, and performance optimization

Nice to have Skills :

• Machine Learning Data Engineering and Feature Stores.
• Power BI Semantic Models and Analytics Enablement.
• Docker and Kubernetes.
• Strong experience working with Linux-based operating systems (RHEL, CentOS, Rocky Linux, Ubuntu
• Cloud platforms (Azure, AWS, or GCP)

Key Responsibilities :

Core Accountabilities

• Design, build, and manage enterprise data products and datasets in accordance with data architecture, governance, security, and data-sharing standards.
• Develop and maintain scalable data ingestion, integration, and transformation pipelines using batch, real-time, and streaming processing patterns.
• Engineer and optimize data solutions across the enterprise data platform, ensuring performance, reliability, scalability, and reusability.
• Design and maintain logical and physical data models to support analytics, AI, and business consumption.
• Implement data quality, validation, monitoring, and observability frameworks to ensure trusted and reliable data assets.
• Administer and support platform services, including orchestration, metadata management, data cataloging, lineage, and access controls.
• Develop APIs, data virtualization, and data-sharing services to enable secure and efficient access to enterprise data.
• Monitor, troubleshoot, and optimize platform operations, data pipelines, and resource utilization to ensure service availability and operational excellence.
• Maintain technical documentation, standards, and operational runbooks to support governance, compliance, and knowledge transfer.

Supporting Accountabilities

• Comply with enterprise data management, information security, governance, and software development policies and procedures.
• Participate in audits, compliance reviews, and assessments related to data platform operations and data governance.
• Support the preparation of operational, service performance, and data quality reports, as required.
• Identify and elevate data quality, security, compliance, and operational risks, recommending corrective actions where appropriate.