Devops Engineer - Data Automation

Don Nelson Recruitment Limited; Hong Kong; On-site; HKD 600,000 - 1,000,000; Full time

Job summary

Don Nelson Technology Limited, in partnership with NP Solution Limited, seeks an experienced Azure Databricks platform engineer to design secure, scalable environments for Hong Kong insurance data workloads. You will build pipelines, automate provisioning with Terraform, and collaborate across architects, engineers, and security teams to ensure seamless integration with legacy systems and cloud sources.

The role focuses on performance tuning, cost control, and regulatory compliance in a

Qualifications

• Bachelor's degree in Computer Science, IT, or a related discipline.
• 5+ years in Azure cloud platform design and administration with Databricks.
• Strong experience provisioning and managing Databricks clusters and workspaces.
• Experience integrating Azure Data Factory and Data Lake Storage Gen2 with Databricks.
• Proficiency with Terraform for infrastructure automation.
• Familiarity with Informatica tools for data governance and cataloguing.
• Deep understanding of security protocols, monitoring, and cost control.

Responsibilities

• Design and manage Azure Databricks environments for enterprise data workloads.
• Collaborate with architects, engineers, and security teams to define standards.
• Automate provisioning with Terraform and CI/CD using GitHub Actions.
• Monitor performance, scale resources, and optimize pipelines and analytics.
• Ensure RBAC, encryption, managed identities, and regulatory compliance.
• Integrate Informatica for metadata and data lineage and support audits.
• Maintain thorough infrastructure documentation and incident response processes.

Skills

Azure Databricks

Terraform

GitHub Actions

CI/CD

Security

Data governance

Cost control

Performance tuning

Education

Bachelor's degree in Computer Science, IT, or related

Tools

Azure Data Factory

Data Lake Storage Gen2

Informatica

Key Vault

NSG

Job description

Design & Manage Platforms: Build secure, scalable, and high-performance Azure Databricks environments tailored for enterprise-wide insurance data transformation.

Cross-Team Collaboration: Work closely with architects, engineers, and security teams to define infrastructure standards and ensure seamless integration with legacy systems, cloud sources, and third‑party platforms.

Infrastructure as Code: Use tools like Terraform to automate provisioning and configuration of Azure and Databricks resources, aligning with DevOps best practices.

Automation & Monitoring: Implement GitHub Actions to automate environment deployment, monitoring, and incident response for greater consistency and efficiency.

Performance Optimization: Monitor platform health and resource usage, anticipate scaling needs, and fine‑tune performance for data pipelines and analytics workloads.

Security & Compliance: Enforce enterprise and regulatory standards such as RBAC, managed identities, encryption, PDPO, GDPR, and insurance‑specific policies.

Data Governance Integration: Oversee integration of Informatica tools for metadata management, data lineage tracking, and compliance validation.

Documentation: Maintain detailed records of infrastructure architecture, network topology, security settings, and operational procedures to support audits and team handovers.

Troubleshooting & Reliability: Diagnose infrastructure issues, perform root cause analysis, and drive continuous improvements for platform stability.

Innovation & Enhancement: Stay updated on Azure, Databricks, and DevOps advancements, recommending upgrades to improve platform capabilities and support business goals in Hong Kong's life and general insurance sectors.

Requirements

Education

Bachelor's degree in Computer Science, IT, or a related discipline.

Experience

5+ years in Azure cloud platform design and administration, with a focus on Azure Databricks for insurance data workloads.

Technical Expertise

Strong skills in provisioning and managing Databricks clusters and workspaces for various data types.

Experience integrating Azure Data Factory and Data Lake Storage Gen2 with Databricks.

Proficient in Terraform for infrastructure automation.

Familiarity with Informatica tools for data governance and cataloguing.

Deep understanding of security protocols (RBAC, NSG, managed identities, Key Vault), monitoring, and cost control in regulated environments.

Hands-on experience with GitHub Actions for CI/CD automation.

Proven ability in incident response and performance tuning for critical data platforms.

Soft Skills

Excellent documentation, collaboration, and communication abilities to support both technical and business stakeholders.

**This position is offered through a strategic partnership between Don Nelson Technology Limited and NP Solution Limited.