Senior Data Platform Engineer

Confidential; Lisboa; On-site; EUR 80,000 - 110,000; Full time

Job summary

Confidential in Lisbon seeks a Senior Data Platform Engineer to design and evolve a modular data platform built on Databricks. You will own ingestion, governance, data modeling, and data products, building reusable capabilities and enabling trusted data access at scale.

You will lead ingestion frameworks, implement data protection measures for GDPR compliance, and drive platform standards across CI/CD, IaC, and observability while mentoring peers.

Qualifications

• 6+ years in data engineering with production Databricks Lakehouse experience
• Strong Spark and Delta Lake experience
• Experience with Unity Catalog or equivalent
• Data anonymisation/pseudonymisation in regulated environments
• Canonical data modelling (data vault, dimensional modelling)
• APIs (REST/GraphQL) design and versioning

Responsibilities

• Develop and evolve modular ingestion patterns for batch, API-based, and event-driven sources
• Build and optimize Databricks-based data pipelines across Bronze/Silver/Gold
• Implement data governance: access controls, lineage, cataloguing, privacy compliance
• Define data quality standards, contracts, monitoring and alerting across layers
• Collaborate with domain teams to integrate source data into the canonical model
• Champion Data as a Product: discoverability, self-service, versioning, contracts
• Mentor engineers and align with governance, security and architecture teams

Job description

We are building a modular, enterprise-grade data platform powered by Databricks, designed to deliver trusted, governed data through multiple consumption channels, including analytics, APIs, streaming, and knowledge graph solutions. The platform is built on a shared canonical data model and emphasizes scalability, governance, privacy, and a Data as a Product mindset.

As a Senior Data Platform Engineer, you will play a key role in designing and evolving the platform across ingestion, governance, data modeling, and data products. This is a hands-on, high-ownership role focused on building reusable platform capabilities, enabling trusted data access at scale, and driving engineering best practices across the data ecosystem.

Responsibilities

• Develop and maintain modular ingestion patterns supporting batch, API-based, and event-driven data sources (e.g., JDBC, SaaS/ERP connectors, Kafka, CDC).
• Establish reusable ingestion frameworks (metadata-driven when possible) that enable rapid and consistent onboarding of new data sources.
• Build and optimize Databricks-based data pipelines (Spark, Delta Live Tables/Lakeflow) across the medallion architecture (Bronze, Silver, Gold).
• Implement and evolve data governance capabilities, including access controls, lineage, cataloging, auditing, data classification, and privacy compliance.
• Define and enforce data quality standards, data contracts, monitoring, and alerting across ingestion and transformation layers.
• Implement data protection and privacy solutions, including masking, tokenization, pseudonymization, and other appropriate anonymization techniques to meet regulatory requirements (e.g. GDPR).
• Build, own, and evolve data products aligned to a canonical data model, ensuring clear ownership, SLAs and documentation.
• Collaborate with domain teams to integrate source data into the canonical model while preserving domain-specific nuance.
• Champion a Data as a Product approach: discoverability, self-service, versioning, and consumer-facings contracts.
• Build and maintain data consumption and serving layers, including BI/analytics datasets, APIs (REST/GraphQL), and use case-driven extensions such as streaming outputs or knowledge graph representations.
• Drive technical standards and platform direction across CI/CD, Infrastructure as Code, testing, observability, and cost optimization
• Mentor engineers and partner with governance, security, architecture, and business stakeholders.

Requirements

• Around 6+ years in data engineering, with 2+ years designing/operating Databricks-based Lakehouse platforms at production scale.
• Hands-on expertise with Apache Spark and Delta Lake; experience with Delta Live Tables/Lakeflow Declarative Pipelines is a strong plus.
• Practical experience with Unity Catalog (or equivalent) for access control, lineage, and cataloguing.
• Demonstrated experience implementing data anonymisation/pseudonymisation techniques in a regulated environment.
• Solid understanding of canonical/conformed data modelling (e.g., data vault, dimensional modelling, domain-driven design for data).
• Track record exposing data via APIs (REST/GraphQL) - designing schemas, contracts, and versioning for external/internal consumers.
• Proficient in Python and/or Scala, SQL, and infrastructure-as-code (Terraform preferred).
• Strong grasp of data governance frameworks, data privacy regulation (GDPR/CCPA), and data security practices.
• Experience with cloud platforms (Azure, AWS, or GCP) and their native data/event services.

Nice to have

• Exposure to Structured Streaming or event-driven ingestion (Kafka, CDC tools such as Debezium) - not a core requirement, but useful as the platform matures.
• Familiarity with knowledge graph concepts (e.g., RDF/property graphs, Neo4j, graph layers on Spark) - a plus rather than an expectation.
• Experience with data mesh or federated data product architectures.
• Experience with metadata management/catalog tools beyond Unity Catalog (e.g., OpenMetadata, Collibra, Purview).
• Exposure to MLOps or feature store patterns on Databricks.
• Relevant certifications (Databricks Data Engineer Professional, cloud architecture certs).