Senior Databricks Data Engineer - Lakehouse & CI/CD

Intellias; Portugal; On-site; EUR 70,000 - 100,000; Full time

Benefits offered by this job

Health Insurance

Annual Leave 22 days

Paid Sick Leave

Permanent Contract

Modern Porto Office

Snacks & Drinks

Discount Platform

Job summary

Intellias is hiring a Senior Data Engineer to join a delivery team for a large-scale enterprise data platform migration for a financial client. You will write production code weekly, own architecture for your workstream, and mentor a mid-level engineer.

You will build medallion pipelines on Databricks, register data in Unity Catalog, ensure data quality, and deploy via CI/CD in a Lakehouse environment. This role is based in Porto with a permanent contract and office support.

Qualifications

• Minimum 6+ years in data engineering with recent 3+ years on Databricks in production.
• Experience with medallion/lakehouse architecture and Delta Lake.
• Strong Python, PySpark and SQL; proficient in Spark UI/Photon performance tuning.
• Experience with streaming ingestion (Kafka) and CDC patterns.
• Experience with data quality at scale and lineage/cataloguing.
• CI/CD for data platforms and IaC (Terraform or equivalent).
• Fluent English for collaboration across teams.

Responsibilities

• Deliver end-to-end data domain pipelines on Azure Databricks.
• Design Bronze/Silver/Gold pipelines using Lakeflow, Auto Loader, and Delta Lake.
• Register curated data products in Unity Catalog with tags and access policies.
• Implement data quality gates and refine rules with stewards.
• Package and deploy work using Databricks Asset Bundles via CI/CD.
• Write automated tests and documentation; collaborate with architecture team.
• Explore AI tooling integration (Claude Code, Databricks Assistant) for automation.

Skills

Databricks

Lakehouse

Python

PySpark

SQL

Kafka

Data Quality

CI/CD

Terraform

English

Tools

Delta Lake

Unity Catalog

Lakeflow

Auto Loader

Job description

Intellias is hiring a Senior Data Engineer to join a delivery team for a large-scale enterprise data platform migration for a financial client. You will write production code weekly, own architecture for your workstream, and mentor a mid-level engineer.

You will build medallion pipelines on Databricks, register data in Unity Catalog, ensure data quality, and deploy via CI/CD in a Lakehouse environment. This role is based in Porto with a permanent contract and office support.