Azure Data Engineer

Landmark Group; Dubai; On-site; AED 150,000 - 190,000; Full time

Job summary

Landmark Group in Dubai seeks an experienced Azure Data Engineer to join the Data Team in Landmark Digital. You will design, deploy and optimize data solutions using Azure Data Factory, Data Lake, Databricks, Synapse and Power BI to drive data-driven decision making.

You will work on large-scale data pipelines (>50TB), ensure data quality, governance and security, and collaborate with cross-functional teams to advance the retail group's data strategy.

Qualifications

• 6–8 years of experience designing, deploying and optimizing data solutions on Azure.
• Proficiency in Python, SQL or Scala for data manipulation and analysis.
• Strong knowledge of data modelling, warehousing, governance and security.
• Experience building ETL pipelines with Azure Data Factory and Azure Data Lake.
• Experience with Azure Databricks and Synapse for large-scale analytics.

Responsibilities

• Design, implement and optimize Azure-based data solutions and pipelines.
• Develop data models, warehousing concepts and data integration strategies.
• Ensure data quality, governance, security and compliance across platforms.
• Collaborate with cross-functional teams to deliver data-driven insights.
• Monitor performance, troubleshoot issues and optimize cost and scalability.

Skills

Python

SQL

Scala

Azure services

Data modeling

Data governance

ETL

Education

Bachelor's degree in CS/Engineering or related

Tools

Azure Data Factory

Azure Databricks

Azure Synapse

Azure Data Lake Storage

Power BI

SQL Server/Databases

Job description

Landmark Group – Introduction

The Landmark Group began its journey in 1973 with one store in Bahrain and has grown into one of the largest retail and hospitality conglomerates in the Middle East, Africa and India. Currently the Group operates over 2,200 outlets, encompassing over 30 million square feet across 21 countries.

Job Purpose

Landmark Group is looking for a Azure Data Engineer to join the Data Team in Landmark Digital.

As a Data Engineer, you will take a pivotal role in designing, deploying, and optimizing data solutions leveraging Azure's powerful ecosystem. Your expertise will drive the development of efficient data pipelines using Azure Data Factory, ensuring data quality and management in Azure Data Lake. Additionally, you'll utilize your proficiency in complex stored procedures within Azure Databricks or Synapse. Collaborating cross-functionally, you'll contribute to data-driven initiatives, harnessing Azure Synapse Analytics to provide insightful intelligence that shapes informed decision-making.

Job Description

• Responsible for the design, implementation, and optimization of data solutions as an expert Azure Data Engineer with 6-8 years of experience.
• Proficiency in programming languages like Python, SQL, or Scala for data manipulation, transformation, and analysis
• Expertise in data modelling, data warehousing, and data integration concepts.
• Sound knowledge of data governance practices, data quality management, and data security principles.
• Develop efficient ETL pipelines using Azure Data Factory and ensure data quality and storage via Azure Data Lake.
• Utilize your proficiency in complex stored procedures within Azure Databricks to enable advanced analytics on data volumes exceeding 50TB.
• Collaborate cross-functionally to contribute to data-driven initiatives, leveraging Azure Synapse Analytics to provide actionable insights.
• Apply your deep understanding of the retail domain to enhance data strategies and solutions for retail-specific challenges.
• Play a pivotal role in shaping our organization's data-driven journey, driving innovation through data analytics and insights.
• Optimize data storage, processing and retrieval mechanisms for performance, cost, and scalability using data storage services (such as Azure Data Lake Storage, Azure SQL Database, etc.), data processing services (such as Azure Data Bricks, Azure Synapse, etc.) and data visualization (PowerBI, Qlik, etc.) & integration services (Data API builder, logic apps, etc.)
• Monitor and troubleshoot data platform performance, identify and resolve issues, and provide recommendations for continuous improvement.
• Collaborate with DevOps teams to automate deployment, configuration, and monitoring processes using Azure DevOps, PowerShell, or other relevant tools.
• Stay up to date with the latest trends and advancements in cloud data services and provide recommendations on adopting new technologies or features to enhance the data platform.
• Document technical designs, procedures, and guidelines for data platform engineering and operations

Knowledge, Skills & Experience

Relevant Job Experience

• Bachelor's degree in Computer Science, Engineering, or a related field. Advanced degree preferred.
• Proven 6-12 years experience in designing, implementing solutions on the Azure ecosystem including expertise in Azure services such as Azure Data Factory, Azure Databricks, Azure Machine Learning, Azure Data Lake Storage, and Azure SQL Database, Azure Synapse, PowerBI etc...
• Experience with big data technologies such as Apache Spark, Hadoop, or similar frameworks.
• Solid understanding of cloud computing concepts and experience with cloud infrastructure management and provisioning.
• Solid understanding of network security concepts and technologies (such as firewalls, VPNs, intrusion detection/prevention systems, etc.) and data security concepts and technologies (such as access controls, encryption, observability, privacy laws/regulations, etc.)
• Experience in a Retail setup is preferred
• Azure certifications such as Azure Data Engineer Associate or Azure Solutions Architect Expert is preferred

Required Leadership Skills

The position will require someone with the following:

• Strategic Planning
• Communication and Collaboration
• Problem Solving Skills