Azure Engineer

CLOUD KINETICS CONSULTING PTE. LTD.; Singapore; On-site; SGD 80,000 - 110,000; Full time

Job summary

A leading cloud solutions provider in Singapore is seeking an experienced Azure Data Engineer. The role involves designing and deploying Azure data platform solutions, building data pipelines, and implementing various Azure services such as Azure Synapse Analytics and Databricks. Strong hands-on experience with Azure tools, data modeling, and cloud architecture is essential, along with proficiency in programming languages like Python. Microsoft Azure certifications are preferred. Competitive compensation and growth opportunities are offered.

Qualifications

• 4+ years of experience working with Microsoft Azure cloud services.
• Strong hands-on experience with Azure Synapse Analytics and Data Factory.
• Familiarity with Azure OpenAI service and integration patterns.
• Proficiency in data processing using Python or PySpark.
• Experience with Infrastructure as Code tools.

Responsibilities

• Design and deploy Azure data platform solutions including Azure Synapse Analytics.
• Build and maintain data ingestion pipelines using Azure Data Factory.
• Implement and manage Azure Databricks workspaces for data processing.
• Monitor data platform resources using Azure Monitor and Log Analytics.
• Implement backup and disaster recovery for critical data assets.

Skills

Microsoft Azure cloud services

Azure Synapse Analytics

Azure Data Factory

Azure Data Lake Storage

Apache Spark

Data engineering

Data warehousing concepts

Python

PowerShell

Terraform

Job description

Key Responsibilities:

Design and deploy Azure data platform solutions including Azure Synapse Analytics, Azure Data Factory, Azure Data Lake Storage (ADLS Gen2), and Databricks/Apache Spark environments.

Build and maintain data ingestion pipelines using Azure Data Factory and Azure

Synapse pipelines to support batch and real-time data processing. Implement and manage Azure Databricks workspaces and Apache Spark clusters for large-scale data processing and analytics workloads.

Configure and optimize Azure Synapse Analytics dedicated SQL pools, serverless SQL pools, and Spark pools for data warehousing and analytics. Deploy and manage Azure Machine Learning workspaces, compute instances, and ML pipelines for model training and deployment.

Implement Azure OpenAI service integrations for generative AI and LLM-based applications within the data platform. Design and implement data lake architecture following medallion architecture (bronze, silver, gold layers) for data organization and governance.

Monitor data platform resources using Azure Monitor, Log Analytics, and service-specific monitoring tools to ensure optimal performance and availability. Manage Azure Active Directory, role-based access control (RBAC), and data security policies including encryption, network security, and private endpoints.

Troubleshoot and resolve L2 escalations related to data pipeline failures, Spark job performance issues, and data platform infrastructure. Implement backup, disaster recovery, and business continuity solutions for critical data assets.

Build and maintain Azure DevOps pipelines for automated deployment of Terraform configurations across development, staging, and production environments.

Required Qualifications:

4+ years of experience working with Microsoft Azure cloud services, with focus on data and analytics platforms.

Strong hands-on experience with Azure Synapse Analytics, Azure Data Factory, and Azure Data Lake Storage.

Proficiency with Apache Spark and distributed data processing concepts. Experience deploying and managing Azure Databricks environments.

Knowledge of Azure Machine Learning services and ML lifecycle management.

Familiarity with Azure OpenAI service and integration patterns.

Understanding of data warehousing concepts, ETL/ELT processes, and data modeling. Experience with data lake architecture patterns and best practices.

Proficiency in PowerShell, Azure CLI, Python, or PySpark for automation and data processing. Understanding of networking concepts including private endpoints, VNets, NSGs, and service endpoints for secure data platform deployments.

Experience with Infrastructure as Code tools like ARM templates, Bicep, or Terraform.

Microsoft Azure certifications such as AZ-104 (Azure Administrator), DP-203 (Azure Data Engineer), or AZ-305 (Azure Solutions Architect) preferred.