DataOps / DevOps Engineer

Epergne Solutions; Abu Dhabi; On-site; AED 279,000 - 334,800; Full time

Job summary

A technology solutions provider in Abu Dhabi is seeking a skilled DataOps/DevOps Engineer to oversee CI/CD practices across Databricks environments. The ideal candidate should have over 5 years of hands-on experience implementing CI/CD in data platforms, with expertise in Databricks workflows and Git-based version control. Responsibilities include designing CI/CD pipelines, monitoring job performance, and maintaining operational stability to ensure secure and efficient data management.

Qualifications

• 5+ years of experience in data platform CI/CD practices.
• Hands-on expertise in Databricks and deployment automation.
• Strong knowledge of Git-based workflows and PR governance.

Responsibilities

• Design and maintain CI/CD pipelines for Databricks.
• Implement monitoring and incident management for jobs.
• Standardize secrets and configuration management.

Skills

CI/CD implementation

Databricks Workflows

Git-based version control

Azure security

Monitoring and alerting practices

DevOps best practices

Job description

Overview

Own the deployment, promotion, automation, and operational stability of Databricks environments, ensuring secure and reliable CI/CD practices across data and AI workloads.

Role

Job Role: DataOps / DevOps Engineer

Job Location: Abu Dhabi, UAE

Experience: 5+ Years

Role Summary

Own the deployment, promotion, automation, and operational stability of Databricks environments, ensuring secure and reliable CI/CD practices across data and AI workloads.

Key Responsibilities

• Design and maintain CI/CD pipelines for Databricks jobs, notebooks, workflows, and configurations
• Implement pull request governance with mandatory reviews, automated validations, and controlled merges
• Manage environment promotion (Dev, Test, Prod) with approval gates and change controls
• Define workflow orchestration standards including retries, dependencies, and failure handling
• Enable versioning, rollback mechanisms, and release traceability for platform assets
• Standardize secrets and configuration management using secure key management solutions
• Implement monitoring, alerting, and incident management for jobs and workflows

Secondary Responsibilities

• Understand Spark, streaming, and AI workloads to prevent unsafe automation changes
• Integrate data quality and validation checks into CI/CD pipelines
• Collaborate with platform and AI teams during new feature adoption

Required Skills & Experience

• Strong experience with CI/CD implementation in data platform environments
• Hands-on expertise with Databricks Workflows and deployment automation
• Experience with Git-based version control and PR governance
• Knowledge of Azure security, identity, and key management integration
• Experience with monitoring, alerting, and production support practices
• Strong understanding of DevOps best practices for data and AI systems