Senior Data Engineer (Databricks/PySpark/SQL) - Portugal or Spain Full Remote

HumanIT Digital Consulting; Lisboa; Remoto; EUR 26.000 - 34.000; Tempo pieno

Vantaggi offerti da questo lavoro

15th month salary

Health insurance for family

Birthday off

Mobility program for digital nomads

Work-life balance

Descrizione del lavoro

HumanIT Digital Consulting is seeking a Senior Data Engineer to design and maintain Databricks-based data pipelines. You will work on a Lakehouse platform, implementing Bronze/Silver/Gold layers and integrating data from enterprise systems and APIs.

The role emphasizes PySpark, SQL development, data governance, and collaboration with Data Architects and BI teams. Remote work from Portugal or Spain is available in an international setting.

Competenze

• 4+ years of professional Data Engineering experience.
• 2+ years of hands-on Databricks development experience.
• Strong practical Databricks and PySpark skills for building and maintaining data processing pipelines.
• Strong SQL development and optimisation skills.
• Experience with Azure Data Platform services.
• Proven experience designing and implementing ETL/ELT pipelines.
• Understanding of Lakehouse architecture and Medallion Architecture (Bronze, Silver, Gold).
• Experience integrating data from multiple enterprise systems, APIs and external sources.
• Knowledge of data quality, pipeline reliability, performance optimisation and data security principles.
• Experience using Git and CI/CD practices in data engineering workflows.
• Ability to collaborate with Data Architects, BI/Analytics teams and business stakeholders.
• Professional working proficiency in English.
• Availability to work remotely from Portugal or Spain.

Mansioni

• Design, develop and maintain scalable data ingestion and transformation pipelines on a Databricks-based Lakehouse platform.
• Implement Bronze, Silver and Gold data layers and integrate data from enterprise systems and external sources.
• Develop processing workflows using PySpark and SQL and optimise pipelines for scalability and cost efficiency.
• Translate architecture standards into practical engineering solutions and reusable frameworks.
• Support data governance, lineage, cataloguing and access management, enabling BI/Analytics teams to consume trusted data.
• Participate in Agile ceremonies and contribute to delivery with Git-based version control and CI/CD practices.

Conoscenze

Data Engineering

Databricks

PySpark

SQL

Azure Data Platform

ETL/ELT pipelines

Lakehouse

Medallion Architecture

Git

CI/CD

English

Remote collaboration

Strumenti

Delta Live Tables

Unity Catalog

Microsoft Fabric

Git

Descrizione del lavoro

Senior Data Engineer (Databricks/PySpark/SQL) - Portugal or Spain Full Remote

ABOUT THE OPPORTUNITY

Join an international technology environment focused on building modern, scalable data platforms that enable better analytics and business decision-making. You will contribute to the development of an enterprise Lakehouse platform, working with modern cloud data technologies and collaborating with Data Architects, Analytics teams and business stakeholders.

This opportunity is ideal for a Data Engineer with strong hands‑on Databricks expertise who enjoys designing robust data pipelines, improving data quality and delivering reliable data products in a collaborative, Agile environment.

PROJECT & CONTEXT

You will design, develop and maintain scalable data ingestion and transformation pipelines on a Databricks-based Lakehouse platform. Your work will cover the implementation of Bronze, Silver and Gold data layers, integrating data from enterprise systems, APIs and external sources while ensuring performance, reliability, security and data quality.

You will develop processing workflows using PySpark and SQL, implement reusable engineering frameworks and optimise pipelines for scalability and cost efficiency. Working closely with Data Architects, you will translate established architecture standards into practical engineering solutions.

The role also involves supporting data governance, lineage, cataloguing and access management initiatives, while enabling BI and Analytics teams to consume trusted, business-ready data. You will participate in Agile ceremonies and contribute to delivery through Git-based version control and CI/CD practices.

WHAT WE'RE LOOKING FOR (Required)

• At least 4 years of professional Data Engineering experience.
• At least 2 years of hands‑on Databricks development experience.
• Strong practical skills in Databricks and PySpark for building and maintaining data processing pipelines.
• Strong SQL development and optimisation skills.
• Experience with Azure Data Platform services.
• Proven experience designing and implementing ETL/ELT pipelines, including data ingestion and transformation.
• Understanding of Lakehouse architecture and Medallion Architecture, including Bronze, Silver and Gold data layers.
• Experience integrating data from multiple enterprise systems, APIs and external sources.
• Knowledge of data quality, pipeline reliability, performance optimisation and data security principles.
• Experience using Git and CI/CD practices in data engineering workflows.
• Ability to collaborate effectively with Data Architects, BI and Analytics teams, and business stakeholders.
• Professional working proficiency in English.
• Availability to work remotely from Portugal or Spain.

NICE TO HAVE (Preferred)

• Experience with Delta Live Tables (DLT).
• Familiarity with Unity Catalog and Databricks data governance capabilities.
• Experience with Microsoft Fabric.
• Previous involvement in Data Hub or Enterprise Data Platform programmes.
• Experience developing reusable data engineering frameworks and components.
• Knowledge of data lineage, cataloguing and enterprise data governance practices.
• Experience optimising cloud data workloads for performance and cost efficiency.

Compensation: €2,300 – €3,050/month net, depending on experience and seniority level.

Why HumanIT people stay (4.4 Glassdoor, 89% recommend)

• 15th month salary
• Health insurance covering your family
• Birthday off
• Mobility program for digital nomads
• Real work-life balance

Full benefits https://www.humanit.pt/careers/#perks

What it's really like https://www.humanit.pt/careers/#work-at