Data Engineer

Intellias; Portugal; Presencial; EUR 70 000 - 100 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Health Insurance

Permanent contract

Office in Porto (modern, well-equipped

Snacks and drinks at the office

Discount platform

Resumo da oferta

Intellias is hiring a Senior Data Engineer to join an enterprise data platform migration programme for a financial services client. You will write production code weekly, own architectural decisions, and mentor a paired mid-level engineer.

You will work on a Databricks-based lakehouse, build pipelines, and register curated data products in Unity Catalog while ensuring data quality and governance. This role is based in Porto with a modern office and strong engineering culture.

Qualificações

• 6+ years of data engineering experience with at least 3 years on Databricks in production.
• Deep hands-on experience with medallion/lakehouse architecture, Delta Lake, Unity Catalog, Lakeflow Declarative Pipelines and Databricks Asset Bundles.
• Strong Python, PySpark and SQL, large-scale performance tuning via Spark UI/Photon.

Responsabilidades

• Deliver end-to-end data domain pipelines on Azure Databricks.
• Design Bronze/Silver/Gold pipelines with Lakeflow, Auto Loader and Delta Lake.

Conhecimentos

Python

PySpark

SQL

Mentoring

English fluency

Ferramentas

Databricks

Delta Lake

Unity Catalog

Lakeflow Declarative Pipelines

Auto Loader

Structured Streaming

Terraform

Azure DevOps

GitHub Actions

Descrição da oferta de emprego

We are hiring a Senior Data Engineer to join an Intellias delivery team on a large-scale enterprise data platform migration programme for a financial services client. This is a hands‑on senior role. You will write production code every week, own architectural decisions on your workstream, and mentor a paired mid‑level engineer.

Project Overview:

• Our client is an independent, active global asset manager with over R3 trillion in assets under management. They are executing a firm‑wide data strategy to govern, manage and engineer data as a product across more than 70 business‑owned sub‑domains.
• As part of this, they are migrating to Databricks as the foundational layer of their Enterprise Data Platform, adopting a lakehouse architecture built on open formats, declarative pipelines and Unity Catalog.

Requirements:

• 6+ years of data engineering experience, with at least the last three years on Databricks in production.
• Deep hands‑on experience with medallion / Lakehouse architecture, Delta Lake, Unity Catalog, Lakeflow Declarative Pipelines and Databricks Asset Bundles.
• Strong python, PySpark and SQL, comfortable diagnosing performance issues on large workloads through Spark UI and Photon.
• Real production experience with streaming ingestion (Kafka Structured Streaming, Auto Loader) and CDC patterns.
• Experience implementing data quality at scale (Great Expectations or equivalent), plus lineage, cataloguing and access control.
• CI/CD on data platforms through Azure DevOps or GitHub Actions, with infrastructure as code (Terraform or equivalent).
• Fluent working English.

- Nice to have

• Active Databricks certifications
• Prior exposure to investment management platforms, asset management operations data, market data feeds from major providers
• CDMP DAMA certification or equivalent data governance credential.
• Experience applying agentic engineering tooling to production data engineering (not just personal productivity), including MCP‑connected Databricks or cloud servers.

Responsibilities:

• Perform the end‑to‑end delivery of one or more priority data domains on Azure Databricks.
• Design and build medallion (Bronze / Silver / Gold) pipelines using Lakeflow Declarative Pipelines, Auto Loader, Structured Streaming and Delta Lake with Liquid Clustering.
• Register curated data products in Unity Catalog with the correct tags, masks, row filters, lineage and access policies.
• Implement data quality gates at the Silver‑to‑Gold boundary, and refine rules with domain stewards.
• Package and deploy work using Databricks Asset Bundles through Azure DevOps CI/CD
• Write automated tests and documentation
• Work at AI‑as‑Collaborator level today with a path to AI‑as‑Orchestrator, using Claude Code, Databricks Assistant, GitHub Copilot and MCP‑connected agent tooling to accelerate the recurring parts of pipeline build, test and deploy.

What we offer:

• 22 days of annual leave per year + 3 paid sick leaves;
• Permanent contract;
• Health Insurance (employee);
• Modern, well‑equipped office in the centre of Porto;
• Snacks and drinks at the office;
• Access to discount platform.

Why this position:

Build within a large-scale enterprise data platform migration from day one, designing and delivering the pipelines, ingestion patterns and domain migrations that bring a 70+ sub‑domain data strategy to life. You'll work hands‑on with the leading edge of the Databricks ecosystem on a high‑stakes lakehouse build, applying agentic engineering tooling to accelerate delivery, and working closely with the architecture team to turn platform standards into working, production‑grade data.