Data Engineer

Dicetek LLC; Dubai; On-site; AED 240,000 - 360,000; Full time

Job summary

Dicetek LLC in Dubai seeks a data engineer to design and implement robust data pipelines from diverse sources to storage, optimizing ingestion and transformation for real-time analytics.

You will collaborate with data scientists to translate requirements into technical specs, and ensure data quality with automated validation and monitoring systems. A 4+ year track record and Azure Data Factory/Databricks expertise are essential.

Qualifications

• Bachelor's degree in Computer Science, IT, Data Engineering or related field.
• 4+ years in Data Engineering, Data Integration and Cloud Data Platforms.
• Strong Azure data platform expertise and SQL development skills.
• Experience with data modeling and data warehousing concepts.
• Proven ability to integrate multiple systems via REST APIs and file protocols.

Responsibilities

• Design and implement robust data pipelines for diverse sources to data storage.
• Optimize data ingestion and transformation for real-time analytics.
• Collaborate with data scientists to translate requirements into technical specs.
• Ensure data quality with automated validation and monitoring systems.

Skills

Azure Data Factory

Azure Databricks

Azure Data Lake Storage Gen2

Azure Synapse Analytics

Delta Lake

Python

PySpark

SQL development

REST APIs

SHIR (Self-Hosted Integration Runtime)

Education

Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field

Tools

Databricks Medallion Architecture

Delta Lake implementation

Storage Accounts

Job description

Design and implement robust data pipelines that ensure seamless data flow from various sources to data storage solutions Optimize data ingestion and transformation processes to improve performance and reduce latency for real-time analytics Collaborate with data scientists and analysts to understand data requirements and translate them into technical specifications Ensure data quality and integrity by implementing automated data validation and monitoring systems

Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field.

• 4+ years of experience in Data Engineering, Data Integration, and Cloud Data Platforms.
• Strong expertise in Azure Data Factory, Azure Databricks, Azure Data Lake Storage Gen2, Azure Synapse Analytics, and Storage Accounts.
• Hands-on experience with Databricks Medallion Architecture and Delta Lake implementation.
• Extensive experience building solutions using Python and PySpark.
• Proven experience integrating Microsoft Dynamics 365, Salesforce, Oracle Fusion, SharePoint, REST APIs, SFTP, and database systems.
• Strong understanding of ETL/ELT frameworks, data modeling, and data warehousing concepts.
• Experience with Self-Hosted Integration Runtime (SHIR) configuration and maintenance.
• Strong SQL development and database optimization skills.
• Excellent troubleshooting, analytical, and problem-solving abilities.
• Strong communication, stakeholder management, and documentation skills.