Databricks Data Platform Lead

GCS Recruitment; Sint-Joost-ten-Node; Sur place; EUR 90 000 - 140 000; Plein temps

Résumé du poste

GCS Recruitment is seeking a Databricks Data Lead to provide technical leadership in the design, development, and delivery of modern cloud-based data platforms using Databricks. You will establish data engineering standards and lead technical delivery to ensure scalable, secure, and reliable data solutions aligned with business goals.

The Databricks Data Lead will collaborate with data architects, engineers, analysts, scientists, and cloud teams to enable enterprise data and analytics

Qualifications

• Significant experience in data engineering, data architecture, or cloud data platforms.
• Strong hands-on experience with Databricks and the Lakehouse architecture.
• Strong knowledge of Apache Spark, PySpark, and SQL.
• Experience designing enterprise-scale data pipelines.
• Strong understanding of Delta Lake and modern lakehouse architectures.
• Experience with cloud platforms such as Azure, AWS, or GCP.
• Strong understanding of data warehousing, data lakes, data modelling, and data integration.
• Experience developing both batch and streaming data solutions.
• Knowledge of data governance, security, and access-control principles.
• Experience with CI/CD, automated testing, and DevOps practices.
• Experience with data orchestration tools such as Databricks Workflows, Apache Airflow, Azure Data Factory.

Responsabilités

• Lead the design and implementation of scalable enterprise data platforms using Databricks.
• Provide technical leadership across data engineering, data integration, data transformation, and analytics initiatives.
• Define and implement modern data architecture patterns using the Databricks Lakehouse architecture.
• Design and implement data pipelines using Spark, PySpark, SQL, and Databricks.
• Establish standards and best practices for data ingestion, transformation, processing, storage, and consumption.
• Lead the development of batch and real-time/streaming data pipelines.
• Design and implement data lake and lakehouse solutions using Delta Lake.
• Define approaches for data modelling, data partitioning, optimisation, and storage management.
• Lead implementation of data governance, security, access control, data quality, lineage, and metadata management.
• Work with architects to establish integration patterns between Databricks and enterprise data sources.

Connaissances

Databricks

Lakehouse

Spark

PySpark

SQL

Cloud platforms

CI/CD

DevOps

Data governance

Leadership

Data modelling

Outils

Databricks

Apache Airflow

Azure Data Factory

Terraform

dbt

Description du poste

GCS Recruitment is seeking a Databricks Data Lead to provide technical leadership in the design, development, and delivery of modern cloud-based data platforms using Databricks. You will establish data engineering standards and lead technical delivery to ensure scalable, secure, and reliable data solutions aligned with business goals.

The Databricks Data Lead will collaborate with data architects, engineers, analysts, scientists, and cloud teams to enable enterprise data and analytics