Freelance Senior Databricks Platform Engineer

adesso Belgium; Belgique; Remote; EUR 90,000 - 150,000; Full time

Job summary

adesso Belgium is urgently seeking a Senior Databricks Platform Engineer to design and build robust data platforms. You will implement Lakehouse architectures, Delta Lake, Unity Catalog, and governance across batch and streaming pipelines using PySpark and Databricks Workflows.

The role covers CI/CD for Databricks assets, IaC with Terraform, and cost/performance optimization, in a flexible freelance setup for a 6-month engagement.

Qualifications

• Proven experience in design & build of data platforms.
• Expertise in Databricks architecture and governance.
• CI/CD and IaC for Databricks with Enterprise GitHub or similar.
• Terraform / IaC for cloud infrastructure and deployments.

Responsibilities

• Design & build data platforms using Databricks and lakehouse patterns.
• Implement Delta Lake, Unity Catalog, and data governance.
• Develop robust batch and streaming pipelines with PySpark.
• Set up CI/CD pipelines for notebooks, jobs, and workflows.
• Monitor, optimize, and troubleshoot pipelines for cost and performance.
• Collaborate to define best practices and reusable components.

Skills

Databricks architecture

Unity Catalog

CI/CD for Databricks

Terraform / IaC

Lakehouse design

Data product enablement

Knowledge graph / AI

Mode integration

Tools

Terraform

GitHub

Azure DevOps

Job description

adesso Belgium , who are we?

We are an IT (start-up) provider working under the adesso umbrella with proven track record in different industries. The new Belgian entity of adesso is part of the expansion and growth strategy of the adesso group.

Requirements

Start date: ASAP- Remote work Duration: 6 months Rate: TBD- depending on your location

Senior Databricks Platform engineer

Your Role in projects:

• Design & build data platforms
• Design and implement Lakehouse architectures on Databricks (medallion / bronze-silver-gold patterns, Delta Lake, Unity Catalog).
• Build robust batch and streaming data pipelines using PySpark and Spark SQL, orchestrated via Databricks Workflows / Jobs.
• Model data for analytics use cases (e.g. star schema for BI, curated datasets for data science).

Ingest, transform & govern data

• Ingest data from APIs, files, databases and SaaS sources into data lakes and lakehouses.
• Implement clear and testable transformation logic (data quality checks, deduplication, SCD, CDC, business rules).
• Apply data governance best practices (cataloguing, lineage, RBAC, masking of sensitive data).

Operate & optimize

• Set up CI/CD and deployment pipelines (Azure DevOps / GitHub) for Databricks assets (notebooks, jobs, pipelines, workflows).
• Using Terraform to automated IaaC setup and reusability
• Monitor, optimize and troubleshoot pipelines and clusters for performance, robustness and cost efficiency.
• Help define and implement best practices, coding standards and reusable components within the team.

Your profile - Required technical skills & knowledge:

• Proven experience in:
• Databricks platform architecture
• Unity Catalog and governance
• CI/CD and DevOps for Databricks - Enterprise GitHub
• Terraform / Infrastructure as Code
• Lakehouse architecture
• Data Product Platform enablement
• The Knowledge Graph, ontology and agentic AI are considered a strong plus.
• Unity catalogue, CICD, IAC and terraform skills in mode implementation and integration

Why join adesso Belgium ?

• Work in an international group with strong local expertise in Belgium.
• Collaborate with cross-functional teams within the international data & AI adesso community.
• Flexible freelance setup with the support of a growing adesso Belgium community.

For one of our international customers, we are urgently looking for a senior Databricks Platform engineer.