Azure Data Engineer

Talent Velocity - Your Strategic HR Partner; Dubai; On-site; AED 200,000 - 300,000; Full time

Job summary

A strategic HR services provider in Dubai is seeking a Data Engineer to develop and maintain data pipelines on Microsoft Azure. The ideal candidate should have 8+ years of experience with Azure DE, Microsoft Fabric, and proficiency in PySpark and SQL. Responsibilities include designing data pipelines, collaborating with stakeholders, and ensuring optimized cloud solutions. This is a full-time position that requires strong technical skills and experience in building robust data platforms.

Qualifications

• 8+ years of experience in data engineering.
• Hands-on experience with Microsoft Fabric and data pipeline development.
• Proficiency in PySpark and SQL for data processing.

Responsibilities

• Design and manage data pipelines on Azure for various data sources.
• Develop ingestion frameworks for both batch and real-time data.
• Collaborate with business stakeholders to meet functional requirements.

Skills

Azure DE

Microsoft Fabric

PySpark

SQL

Job description

Overview

The Data Engineer will build and maintain data pipelines, storage layers, data products based on the architecture design and meet organizational requirements. This role involves working closely with end client and team consisting of backend engineers, cloud engineers, and data analysts to enable seamless data integration and building robust data platforms on Azure cloud.

Qualifications

• Must Have Skills : Azure DE, Microsoft Fabric, PySpark, SQL
• Experience : 8+ years
• Notice Period : 15 days to immediate joiner
• Location : Dubai (UAE)

Key Responsibilities

Overall Responsibilities

• Design, develop and manage data pipelines on Microsoft Azure for internal and external data sources.
• Develop ingestion frameworks for batch and real-time data.
• Ensure data pipelines are optimized for reliability and scalability.
• Ensure the scalability, security, and performance of our cloud architecture while aligning with business goals.
• Work closely with solution architect, backend developers, data governance specialists

Technical Responsibilities

• Collaborate with business stakeholders to translate functional requirements into scalable technical solutions.
• Hands-on experience with Microsoft Fabric platform in building end-to-end data solutions.
• Hands-on experience with building pipelines using MS Fabric.
• Build and optimize data lake zones (raw, curated, consumption).
• Proficiency in developing business and dataflow logic for data processing using tools like PySpark, T SQL, Stored Procedures.
• Integrate cloud-based solutions with on-premises systems.
• Implement infrastructure as code (IaC) solutions using Terraform for consistent, automated provisioning.
• In-depth knowledge of Azure architecture components, including compute, storage, networking, and security.
• Optimize cloud storage solutions for performance and cost.
• Establish monitoring and logging solutions using Azure Monitor, Log Analytics, Application Insights, and third-party tools.
• Configure automated scaling solutions to handle workload fluctuations.
• Create and maintain comprehensive architecture documentation, including diagrams, processes, and technical specifications.

Employment details

• Seniority level : Mid-Senior level
• Employment type : Full-time
• Job function : Information Technology
• Industries : Human Resources Services