Data Architect (Databricks)

DISCOVERED; Abu Dhabi; On-site; AED 300,000 - 420,000; Full time

Job summary

DISCOVERED in Abu Dhabi is seeking an experienced Data Architect to design and deliver modern data platforms that support analytics, machine learning and Generative AI initiatives across the business. This hands-on role focuses on the Databricks Lakehouse Platform to ensure data is accessible, reliable and ready for production AI use cases.

You will work closely with Data Science, AI Engineering and Technology teams to implement scalable data solutions, governance, and performance optimisations

Qualifications

• 7+ years of experience in Data Engineering, Data Architecture or Enterprise Data Platform roles.

Responsibilities

• Design and implement scalable data architectures using the Databricks Lakehouse Platform, supporting both analytical and AI workloads.
• Build and optimise ETL/ELT pipelines for large volumes of structured and unstructured data, including real-time and batch processing.
• Partner with AI and Data Science teams to deliver the data capabilities required for production ML, predictive analytics and Generative AI applications.
• Establish best practices across data modelling, governance, quality, security, lineage and metadata management.
• Optimise data platform performance, scalability and cost efficiency across cloud environments.
• Collaborate with engineering and business stakeholders to translate requirements into robust technical solutions and support the integration of AI into enterprise applications.
• Document architecture standards and contribute to the continuous improvement of the organisation’s data platforms and engineering practices.

Skills

Databricks Lakehouse

Python

SQL

PySpark

Data Modeling

Data Governance

Stakeholder Management

Cloud Data Platforms

Education

Bachelor's degree in CS/Engineering/Data

Tools

Databricks

Airflow

dbt

Azure

MLOps

Job description

Discover the Opportunity

We're partnering with a leading financial services organisation in Abu Dhabi that is continuing to expand its Data & AI capabilities.

They're looking for an experienced Data Architect to design and deliver modern data platforms that support advanced analytics, machine learning and Generative AI initiatives across the business.

This is a hands-on architecture role with a strong focus on Databricks. You'll work closely with Data Science, AI Engineering and Technology teams to ensure that data is accessible, reliable and ready to support production AI use cases.

Discover the Responsibilities

• Design and implement scalable data architectures using the Databricks Lakehouse Platform, supporting both analytical and AI workloads.
• Build and optimise ETL/ELT pipelines for large volumes of structured and unstructured data, including real-time and batch processing.
• Partner with AI and Data Science teams to deliver the data capabilities required for production ML, predictive analytics and Generative AI applications.
• Establish best practices across data modelling, governance, quality, security, lineage and metadata management.
• Optimise data platform performance, scalability and cost efficiency across cloud environments.
• Collaborate with engineering and business stakeholders to translate requirements into robust technical solutions and support the integration of AI into enterprise applications.
• Document architecture standards and contribute to the continuous improvement of the organisation’s data platforms and engineering practices.

Discover the Requirements

7+ years of experience in Data Engineering, Data Architecture or Enterprise Data Platform roles.

Extensive hands-on experience designing and delivering enterprise solutions using Databricks and modern lakehouse architectures.

Proven experience supporting the delivery of AI and Machine Learning use cases into production, working closely with Data Science and AI Engineering teams.

Strong expertise in Python, SQL and PySpark, with experience building scalable.

ETL/ELT pipelines using tools such as Airflow, dbt or similar.

Strong understanding of distributed computing, data modelling, big data processing and performance optimisation.

Experience with cloud-based data platforms, ideally Azure, and familiarity with MLOps and the data requirements of production AI systems.

Experience across data governance, quality, security, lineage and metadata management.

Excellent problem-solving and stakeholder management skills, with the ability to communicate technical concepts clearly.

Bachelor's degree in Computer Science, Engineering, Information Systems or a related discipline.