Cloud Engineer Azure Databricks

Remotedxb; Dubai; On-site; AED 260,000 - 420,000; Full time

Job summary

Remotedxb seeks an experienced Cloud Infrastructure/Platform Engineer to design and run the Azure foundation for its data and AI platform in Dubai. You will implement secure networking, build reusable Terraform modules for Databricks and ADLS Gen2, and automate ML workloads with MLflow.

Strong Azure, Terraform, and Databricks skills are essential in this senior role. You will drive cost visibility, policy-based security, and enterprise-scale data engineering infrastructure, including Unity

Qualifications

• 6+ years in cloud infrastructure or platform engineering.
• 4+ years focused on Microsoft Azure.
• Expert-level hands-on experience with Terraform.
• Demonstrable experience in Databricks administration, including Unity Catalog and cluster policies.
• Deep knowledge of Azure Networking, Entra ID, and RBAC.
• Experience with ADLS Gen2 design at enterprise scale.
• Platform-side experience with Azure Data Factory.
• Proficiency in Python, PowerShell, and/or Bash.

Responsibilities

• Design, implement, and administer the Azure foundation for the data and AI platform.
• Author and maintain reusable Terraform modules for Databricks, ADLS Gen2, and Azure ML.
• Implement private networking, hub-and-spoke topology, and security posture.
• Manage ADLS Gen2 storage account architecture and access patterns.
• Oversee Azure Data Factory integration runtime provisioning and deployment automation.
• Provision infrastructure for machine learning workloads including GPU quota and MLflow integration.
• Build monitoring layers using Azure Monitor and Log Analytics.
• Manage FinOps, cost visibility, and capacity planning for DBU and storage spend.
• Establish platform standards and reference configurations for Databricks administration.

Skills

6+ years in cloud infra

4+ years Azure

Python

PowerShell

Bash

Tools

Terraform

Databricks

Unity Catalog

Cluster policies

Azure Data Factory

Azure Monitor

Log Analytics

AKS

Job description

Responsibilities

• Design, implement, and administer the Azure foundation for the data and AI platform
• Author and maintain reusable Terraform modules for Databricks, ADLS Gen2, and Azure ML
• Implement private networking, hub-and-spoke topology, and security posture
• Manage ADLS Gen2 storage account architecture and access patterns
• Oversee Azure Data Factory integration runtime provisioning and deployment automation
• Provision infrastructure for machine learning workloads including GPU quota and MLflow integration
• Build monitoring layers using Azure Monitor and Log Analytics
• Manage FinOps, cost visibility, and capacity planning for DBU and storage spend
• Establish platform standards and reference configurations for Databricks administration

Requirements

• 6+ years in cloud infrastructure or platform engineering
• 4+ years focused on Microsoft Azure
• Expert-level hands‑on experience with Terraform
• Demonstrable experience in Databricks administration, including Unity Catalog and cluster policies
• Deep knowledge of Azure Networking, Entra ID, and RBAC
• Experience with ADLS Gen2 design at enterprise scale
• Platform-side experience with Azure Data Factory
• Proficiency in Python, PowerShell, and/or Bash

Preferred Qualifications

• Experience with Databricks Asset Bundles and workspace-as-code patterns
• Experience leading Unity Catalog migrations
• Knowledge of Azure Machine Learning and MLflow
• Experience with Kubernetes, AKS, and containerized workloads
• Familiarity with Policy as Code (Azure Policy, OPA, Sentinel)
• Experience with Event Hubs, Kafka, or Stream Analytics
• Formal FinOps practice experience