Azure Data Engineer

UST; Bayan Lepas; On-site; MYR 120,000 - 180,000; Full time

Job summary

UST is seeking a Lead Data Engineer to design, develop, and manage end-to-end data solutions on the Microsoft Azure platform. You will enable reliable data ingestion, transformation, storage, and analytics to support business operations and strategic decisions.

The ideal candidate will bring extensive experience with Azure Data Factory, Synapse Analytics, and Databricks, translating business requirements into scalable data architectures.

Qualifications

• 8+ years hands-on experience with Azure data engineering and ETL/ELT design.
• Experience with data warehousing, dimensional modeling, and cloud data platforms.
• Strong REST API integration, data quality, and monitoring capabilities.

Responsibilities

• Design and implement end-to-end data pipelines on Azure.
• Develop data warehouses and analytics models using Azure Synapse.
• Collaborate with cross-functional teams to ensure data quality.
• Monitor, troubleshoot, and optimize production data pipelines.

Skills

Azure Data Factory

Azure Synapse Analytics

ADLS Gen2

Azure Databricks

Apache Spark

REST API integrations

ETL/ELT

Data Warehousing

Dimensional Modeling

Star Schema

Snowflake Schema

Metadata-driven architecture

Azure DevOps

Azure Functions

Azure Logic Apps

Microsoft Fabric

DP-203

Job description

We are seeking a highly skilled and passionate Lead Data Engineer to join our growing Data & Analytics team. In this role, you will be responsible for designing, developing, and managing end-to-end data solutions on the Microsoft Azure platform. You will play a critical role in enabling reliable data ingestion, transformation, storage, and analytics capabilities that support business operations, reporting, and strategic decision-making.

The ideal candidate will have strong experience in Azure Data Engineering technologies, data warehousing, ETL/ELT development, and cloud-native data architectures, with the ability to translate business requirements into scalable and efficient data solutions.

Key Responsibilities

• Design and develop end-to-end data pipelines using Azure Data Factory (ADF).
• Ingest data from REST APIs, SharePoint, Excel, and ADLS Gen2.
• Build scalable ETL/ELT solutions using ADF Mapping Data Flows and metadata-driven frameworks.
• Develop and optimize data models, data warehouses, and analytics solutions using Azure Synapse Analytics.
• Leverage Azure Databricks and Apache Spark for large-scale data processing.
• Implement data quality, validation, and monitoring processes to ensure reliable datasets.
• Collaborate with business and technical teams to deliver robust data solutions.
• Monitor, troubleshoot, and enhance production data pipelines.

Required Skills

• 8+ Years of Strong hands-on experience with Azure Data Factory (ADF), including pipeline orchestration and Mapping Data Flows.
• Experience with Azure Synapse Analytics, ADLS Gen2, and Azure Databricks.
• Expertise in REST API integrations, authentication, pagination, and error handling.
• Strong knowledge of ETL/ELT, data pipeline design, incremental loading, and data validation.
• Experience in Data Warehousing, Dimensional Modeling, Star Schema, and Snowflake Schema.
• Good understanding of metadata-driven architecture and scalable cloud data platforms.

Preferred Skills

• Experience with Azure DevOps (CI/CD pipelines).
• Hands-on exposure to Azure Functions and Azure Logic Apps.
• Knowledge of Microsoft Fabric and modern data architecture patterns.
• Azure certifications such as DP-203 are a plus.