Senior Data Engineer - Azure Data Platform Architect

NorthBay Solutions; United Arab Emirates; Remote; AED 240,000 - 360,000; Full time

Job summary

NorthBay Solutions is seeking a Senior Data Engineer to design, develop, and manage modern data platforms and analytics solutions on Microsoft Azure. The role focuses on building scalable data pipelines, data lakehouse architectures, and enterprise data governance.

The ideal candidate has 8–10+ years of experience with Azure Data Services, Spark, Python/Scala/Java, and large-scale ETL/ELT implementations, enabling advanced BI, AI, and analytics initiatives.

Qualifications

• 8–10+ years of experience in data engineering and enterprise data platform development.
• Strong hands-on experience with Microsoft Azure Data Services (ADF, Synapse, ADLS Gen2, Azure SQL).
• Experience building and managing large-scale ETL/ELT pipelines and data integration solutions.
• Strong proficiency in SQL, query optimization, and database performance tuning.
• Hands-on experience with PySpark, Apache Spark, and distributed data processing frameworks.
• Strong programming skills in Python, Scala, or Java.
• Experience with dimensional modeling and modern lakehouse architectures.
• Experience with structured, semi-structured, and unstructured data.
• Knowledge of data governance, data quality, metadata management, and security best practices.

Responsibilities

• Design, develop, and maintain scalable data platforms and data pipelines on Microsoft Azure.
• Build and optimize batch and real-time data ingestion frameworks from multiple data sources.
• Design and implement data lake, data warehouse, and lakehouse architectures for analytics.
• Develop and manage ETL/ELT processes using cloud-native data engineering practices.
• Implement data transformation, cleansing, validation, and quality frameworks.
• Collaborate with stakeholders to understand data requirements and deliver scalable solutions.
• Optimize data storage, processing, and query performance across platforms.
• Implement security, governance, monitoring, and compliance across Azure environments.
• Support integration with AI/ML, BI, and enterprise applications.
• Participate in architecture reviews, code reviews, and mentoring.

Skills

Data engineering

Azure data services

SQL

PySpark

Python/Scala/Java

ETL/ELT

Data warehousing

Data governance

CI/CD

REST APIs

Education

Bachelor's degree in Computer Science/Engineering

Tools

Azure Data Factory

Azure Synapse Analytics

Azure Data Lake Storage Gen2

Azure SQL Database

Databricks

Delta Lake

Job description

NorthBay Solutions is seeking a Senior Data Engineer to design, develop, and manage modern data platforms and analytics solutions on Microsoft Azure. The role focuses on building scalable data pipelines, data lakehouse architectures, and enterprise data governance.

The ideal candidate has 8–10+ years of experience with Azure Data Services, Spark, Python/Scala/Java, and large-scale ETL/ELT implementations, enabling advanced BI, AI, and analytics initiatives.