Senior Data Engineer - Databricks Lakehouse Architect

Intellias; Poland; On-site; PLN 240,000 - 360,000; Full time

Job summary

Intellias is hiring a Senior Data Engineer to join a delivery team for a large enterprise data platform migration in the financial sector. You will own architectural decisions, write production code weekly, and mentor a mid-level engineer on the journey to AI-enabled data pipelines.

You will work on Databricks, Delta Lake, Unity Catalog and lakehouse patterns, integrating with Azure DevOps or GitHub Actions for CI/CD and ensuring robust data quality and governance across domains.

Qualifications

• 6+ years of data engineering experience, with at least the last three years on Databricks in production.
• Deep hands-on experience with medallion / Lakehouse architecture, Delta Lake, Unity Catalog, Lakeflow Declarative Pipelines and Databricks Asset Bundles.
• Strong python, PySpark and SQL, comfortable diagnosing performance issues on large workloads through Spark UI and Photon.
• Real production experience with streaming ingestion (Kafka Structured Streaming, Auto Loader) and CDC patterns.
• Experience implementing data quality at scale (Great Expectations or equivalent), plus lineage, cataloguing and access control.
• CI/CD on data platforms through Azure DevOps or GitHub Actions, with infrastructure as code (Terraform or equivalent).
• Fluent working English.

Responsibilities

• Deliver end-to-end data domains on Azure Databricks.
• Design and build medallion pipelines (Bronze/Silver/Gold) using Lakeflow and Delta Lake.
• Register curated data products in Unity Catalog with proper tags, masks and lineage.
• Implement data quality gates and refine rules with domain stewards.
• Package and deploy work via Databricks Asset Bundles through CI/CD pipelines.
• Write automated tests and documentation.
• Collaborate across teams and mentor a mid-level engineer.

Skills

Databricks

Python

PySpark

SQL

Delta Lake

Unity Catalog

Lakeflow Declarative Pipelines

Streaming

CI/CD

Terraform

Azure DevOps

GitHub Actions

Databricks Asset Bundles

Auto Loader

Tools

Databricks

Unity Catalog

Delta Lake

Lakeflow Declarative Pipelines

Auto Loader

Structured Streaming

Databricks Asset Bundles

Terraform

Azure DevOps

GitHub Actions

Job description

Intellias is hiring a Senior Data Engineer to join a delivery team for a large enterprise data platform migration in the financial sector. You will own architectural decisions, write production code weekly, and mentor a mid-level engineer on the journey to AI-enabled data pipelines.

You will work on Databricks, Delta Lake, Unity Catalog and lakehouse patterns, integrating with Azure DevOps or GitHub Actions for CI/CD and ensuring robust data quality and governance across domains.