Remote Senior Data Engineer – Lakehouse & Databricks Expert

Intellias; España; Remote; EUR 90,000 - 140,000; Full time

Job summary

Intellias is hiring a Senior Data Engineer to join a delivery team on a large-scale migration for a financial services client. You will own architectural decisions, write production code weekly, and mentor a mid-level engineer on the team.

The role focuses on Databricks lakehouse architecture, Delta Lake, Unity Catalog, and declarative pipelines, with exposure to CI/CD, IaC and data quality practices in a global asset management context.

Qualifications

• 6+ years of data engineering experience, with at least the last three years on Databricks in production.
• Deep hands-on experience with medallion / Lakehouse architecture, Delta Lake, Unity Catalog, Lakeflow Declarative Pipelines and Databricks Asset Bundles.
• Strong Python, PySpark and SQL, comfortable diagnosing performance issues on large workloads through Spark UI and Photon.
• Real production experience with streaming ingestion (Kafka Structured Streaming, Auto Loader) and CDC patterns.
• Experience implementing data quality at scale (Great Expectations or equivalent), plus lineage, cataloguing and access control.
• CI/CD on data platforms through Azure DevOps or GitHub Actions, with infrastructure as code (Terraform or equivalent).
• Fluent working English.

Responsibilities

• Perform the end-to-end delivery of one or more priority data domains on Azure Databricks.
• Design and build medallion (Bronze / Silver / Gold) pipelines using Lakeflow Declarative Pipelines, Auto Loader, Structured Streaming and Delta Lake with Liquid Clustering.
• Register curated data products in Unity Catalog with the correct tags, masks, row filters, lineage and access policies.
• Implement data quality gates at the Silver-to-Gold boundary, and refine rules with domain stewards.
• Package and deploy work using Databricks Asset Bundles through Azure DevOps CI/CD
• Write automated tests and documentation
• Work at AI-as-Collaborator level today with a path to AI-as-Orchestrator, using Claude Code, Databricks Assistant, GitHub Copilot and MCP-connected agent tooling to accelerate the recurring parts of pipeline build, test and deploy.

Skills

Python

PySpark

SQL

English fluency

Tools

Databricks

Delta Lake

Unity Catalog

Lakeflow Declarative Pipelines

Auto Loader

Structured Streaming

Kafka

Great Expectations

Terraform

Azure DevOps

GitHub Actions

Job description

Intellias is hiring a Senior Data Engineer to join a delivery team on a large-scale migration for a financial services client. You will own architectural decisions, write production code weekly, and mentor a mid-level engineer on the team.

The role focuses on Databricks lakehouse architecture, Delta Lake, Unity Catalog, and declarative pipelines, with exposure to CI/CD, IaC and data quality practices in a global asset management context.