Senior Databricks Specialist/Data Architect | Marc Ellis | Doha, Qatar

Marc Ellis; Doha; Presencial; QAR 437 000 - 619 000; Tempo integral

Resumo da oferta

Marc Ellis in Doha, Qatar seeks a Senior Databricks Specialist / Data Architect to lead enterprise data platforms using Azure Databricks, Spark, and Delta Lake. You will govern data pipelines, implement Unity Catalog, and optimize performance across large datasets.

The role requires 8+ years in data engineering/architecture, 4+ years with Azure Databricks, and strong experience with Unity Catalog and Medallion Architecture. 12-month extendable contract on Marc Ellis payroll.

Qualificações

• Bachelor's degree in Computer Science, Data Science, IT, Software Engineering, or related field.
• 8+ years in data engineering, data architecture, or big data systems.
• 4+ years hands-on experience with Microsoft Azure Databricks.
• Deep knowledge of Unity Catalog, Delta Lake, and Medallion Architecture.

Responsabilidades

• Design and optimize enterprise-scale data platforms on Azure Databricks.
• Build high-performance data pipelines using Spark and PySpark.
• Architect Delta Lake storage and Medallion Architecture (Bronze-Silver-Gold).
• Implement Unity Catalog governance, access controls, and data lineage.
• Integrate with Dynamics 365 CRM and other core systems, APIs, and data sources.
• Develop CI/CD pipelines and IaC using Azure DevOps and Git.
• Tune performance, reduce costs, and troubleshoot complex data issues.

Conhecimentos

Azure Databricks

Apache Spark

Delta Lake

Unity Catalog

PySpark

SQL

Azure Data Factory

Azure Data Lake Storage Gen2

CI/CD pipelines

Azure DevOps

Formação académica

Bachelor's degree in Computer Science or related field

Ferramentas

Azure DevOps

Git

Dynamics 365 CRM

Python

Descrição da oferta de emprego

Position Summary

We are seeking an exceptionally skilled and experienced Senior Databricks Specialist / Data Architect to join Marc Ellis on a 12-month extendable contract basis (long-term under Marc Ellis payroll) in Doha, Qatar. In this critical enterprise data leadership role, you will be responsible for designing, implementing, and optimizing enterprise-scale data platforms using Microsoft Azure Databricks, Apache Spark, and Delta Lake architectures. The ideal candidate will bring 8+ years of experience in data engineering and architecture, at least 4+ years of strong hands-on Azure Databricks expertise, and deep technical familiarity with Unity Catalog and Medallion Architecture. If you possess a distinguished background in big data platform engineering and multi-system integration, we invite you to apply.

Detailed Job Description

As a Senior Databricks Specialist / Data Architect at Marc Ellis in Doha, Qatar, you will serve as the principal technical authority responsible for architecting, governing, and scaling the organization’s enterprise data platform and its integration with Dynamics 365 CRM and other core enterprise systems. Your daily responsibilities will encompass designing robust data pipelines using Azure Data Factory, implementing Delta Lake storage layers, configuring Unity Catalog governance and access controls, and driving performance tuning and cost optimization on Azure Databricks. You will troubleshoot complex production data issues and collaborate with engineering teams to automate CI/CD deployments via Azure DevOps.

Key Responsibilities

• Design, implement, architect, and optimize enterprise-scale data platforms using Microsoft Azure Databricks.
• Build and scale high-performance data pipelines and processing frameworks using Apache Spark and PySpark.
• Architect and manage Delta Lake storage layers and implement Databricks Medallion Architecture (Bronze, Silver, Gold).
• Implement enterprise data governance, access control, and data lineage tracking using Unity Catalog.
• Integrate Azure Databricks seamlessly with APIs, enterprise applications, Dynamics 365 CRM, and auxiliary data sources.
• Design and maintain robust cloud data infrastructure using Azure Data Factory and Azure Data Lake Storage Gen2.
• Apply advanced data modeling, dimensional modeling, and data quality standards across large enterprise datasets.
• Drive continuous performance optimization, cost reduction, and query tuning on Databricks clusters.
• Build automated CI/CD deployment pipelines, version control workflows, and infrastructure as code using Azure DevOps and Git.
• Troubleshoot complex production data issues, memory bottlenecks, and pipeline failures rapidly.

Required Qualifications & Skills

• Bachelor's degree in Computer Science, Data Science, Information Technology, Software Engineering, or a related discipline.
• Minimum of 8+ years of professional experience in data engineering, data architecture, or big data systems.
• Minimum of 4+ years of strong hands-on professional experience with Microsoft Azure Databricks.
• Deep technical proficiency in Apache Spark, PySpark, and Delta Lake data formats.
• Strong experience designing enterprise data platforms, data pipelines, and Databricks Medallion Architectures.
• Advanced practical knowledge of Unity Catalog, data governance, security policies, and access controls.
• Strong SQL skills and proven capability in processing large-volume enterprise datasets.
• Solid background with Azure data services including Azure Data Factory and Azure Data Lake Storage Gen2.
• Experience with CI/CD deployment automation, Git, and Azure DevOps pipelines.
• Mandatory residency/work authorization: Ability to operate in Doha, Qatar.

Nice-to-Have Skills

• Professional certifications such as Databricks Certified Data Engineer Professional or Azure Solutions Architect Expert.
• Prior experience integrating Databricks with enterprise CRM platforms (Dynamics 365, Salesforce).
• Experience implementing FinOps cost-optimization strategies for cloud big data workloads.
• Immediate availability for 12-month contract mobilization in Doha.

Application Information

• Recruiter: Marc Ellis
• Contact Name: Sindhu G. / Recruitment Team
• Email: [e-mail] & [e-mail]
• Phone:
• Application URL:
• Salary/Rate: Competitive market rate
• Deadline: Immediate
• Notice Period: Immediate to 30 Days
• Contract Duration: 12 Months Extendable Contract (Long Term Under Marc Ellis Payroll)

Recruitment Pro Tip

When applying for this Senior Databricks Specialist position with Marc Ellis, ensure your resume explicitly highlights your 4+ years of Azure Databricks experience, your Unity Catalog governance portfolio, and your Medallion Architecture implementation background.