AZURE DATABRICKS DATA ENGINEER (HYBRID)

iTRTech Group; Lisboa; Híbrido; EUR 30 000 - 33 000; Tempo integral

Resumo da oferta

iTRTech Group is seeking a mid-level Azure Databricks Data Engineer for a hybrid role in Lisbon, Portugal. You will design, build and optimize data pipelines using Azure Databricks, dbt and Airflow, ensuring data quality and governance across ingestion, transformation, and deployment stages.

The role requires 3+ years of experience, strong SQL and Python, and English and Portuguese at B2 level. The position offers a hybrid model with two office days per week and flexibility to collaborate with a

Qualificações

• 3+ years of Data Engineering experience required.
• Advanced SQL and strong Python skills required.
• Production experience with Azure Databricks, ADLS Gen2, Delta Lake and Medallion Architecture.
• Strong dbt experience including models, incremental materialisations, tests and troubleshooting.
• Experience with Airflow/Astronomer DAGs, scheduling, retries and monitoring.
• Data ingestion experience with APIs, SFTP/files and JDBC.
• Knowledge of data modelling, SCD Type 2, surrogate keys, schema evolution, reprocessing and reconciliation.
• Experience with Spark/PySpark, Scala, Databricks Notebooks and Jobs.
• Experience with Git, Azure DevOps, Pull Requests and CI/CD.
• Knowledge of Data Quality, Metadata and Data Lineage.
• English and Portuguese at B2 level (spoken and written).
• Availability for hybrid work in Lisbon.

Responsabilidades

• Develop data pipelines using Azure Databricks, ADLS Gen2 and Delta Lake;
• Implement Medallion Architecture and transformation models with dbt;
• Create and maintain Airflow/Astronomer DAGs;
• Ingest data through APIs, SFTP/files and JDBC;
• Develop solutions using SQL, Python, Scala, Spark and PySpark;
• Implement data modelling, quality, metadata and lineage practices;
• Manage automated validations and deployments through Azure DevOps and CI/CD;
• Troubleshoot production issues and perform root-cause analysis;
• Produce technical documentation and communicate with stakeholders.

Conhecimentos

SQL

Python

Azure Databricks

Airflow

dbt

Spark/PySpark

Git

CI/CD

Data modelling

Data quality

Data lineage

SCD Type 2

APIs ingestion

English

Portuguese

Ferramentas

Azure Databricks Notebooks

Delta Lake

Astronomer

Descrição da oferta de emprego

• Azure Key Vault;
• Microsoft Certified: Azure Databricks Data Engineer Associate (DP-750);

AZURE DATABRICKS DATA ENGINEER (HYBRID LISBON)

Portuguese company hires for hybrid position

Location: Lisbon, Portugal

Only candidates already based in Portugal will be considered

Work Model: Hybrid — two days per week at the office, with some flexibility

Language Requirements: English B2 and Portuguese B2

Seniority: Mid-level (3+ years)

Rate Between €2700 - 3000 RV (gross value)

About The Opportunity

You will design, build and maintain enterprise data pipelines using Azure Databricks, dbt and Airflow/Astronomer. The role covers data ingestion, transformation, modelling, orchestration, quality, deployment and production troubleshooting.

Main Responsibilities

• Develop data pipelines using Azure Databricks, ADLS Gen2 and Delta Lake;
• Implement Medallion Architecture and transformation models with dbt;
• Create and maintain Airflow/Astronomer DAGs;
• Ingest data through APIs, SFTP/files and JDBC;
• Develop solutions using SQL, Python, Scala, Spark and PySpark;
• Implement data modelling, quality, metadata and lineage practices;
• Manage automated validations and deployments through Azure DevOps and CI/CD;
• Troubleshoot production issues and perform root-cause analysis;
• Produce technical documentation and communicate with stakeholders.

Mandatory Requirements

• 3+ years of relevant Data Engineering experience;
• Advanced SQL and strong Python skills;
• Production experience with Azure Databricks, ADLS Gen2, Delta Lake and Medallion Architecture;
• Strong dbt experience, including models, incremental materialisations, tests, dependencies and troubleshooting;
• Experience with Airflow/Astronomer, including DAGs, scheduling, retries, backfills and monitoring;
• Data ingestion experience with APIs, SFTP/files and JDBC;
• Knowledge of data modelling, SCD Type 2, surrogate keys, schema evolution, reprocessing and reconciliation;
• Experience with Spark/PySpark, Scala, Databricks Notebooks and Jobs;
• Experience with Git, Azure DevOps, Pull Requests and CI/CD;
• Knowledge of Data Quality, Metadata and Data Lineage;
• English and Portuguese at B2 level;
• Availability for hybrid work in Lisbon.

Nice to Have

• Unity Catalog;
• Microsoft Purview;
• Azure Monitor and Log Analytics;
• Microsoft Fabric;
• Power BI Semantic Models;
• Microsoft Entra ID;
• Azure Key Vault;
• Impala;
• Legacy Data Lakes and pipeline modernisation;
• Tableau;
• Pentaho.

Preferred Certifications

• Microsoft Certified: Azure Databricks Data Engineer Associate (DP-750);
• Databricks Certified Data Engineer Professional;
• dbt Analytics Engineering Certification;
• Astronomer Apache Airflow DAG Authoring Certification;
• Astronomer Apache Airflow Fundamentals Certification.

Ideal Candidate Profile

An autonomous Data Engineer with solid production experience in Azure Databricks, dbt and Airflow. The ideal candidate can independently design, deploy and troubleshoot enterprise pipelines while ensuring data quality, lineage and clear technical documentation.

Questions for Candidates

• How many years of Data Engineering experience do you have?
• What is your production experience with Azure Databricks, ADLS Gen2 and Delta Lake?
• Have you worked with dbt models, incremental materialisations and tests?
• Have you created Airflow/Astronomer DAGs with retries, backfills and monitoring?
• Do you have experience with APIs, SFTP/files and JDBC ingestion?
• Have you implemented SCD Type 2, data quality and lineage controls?
• What is your level of English and Portuguese?
• Are you available to work in Lisbon twice per week?
• What is your availability and expected rate?

CV Keywords

Azure Databricks, Microsoft Azure, ADLS Gen2, Delta Lake, Delta Tables, Medallion Architecture, dbt, Apache Airflow, Astronomer, DAGs, SQL, Python, Scala, Spark, PySpark, Databricks Notebooks, Databricks Jobs, APIs, SFTP, JDBC, Data Modelling, SCD Type 2, Surrogate Keys, Schema Evolution, Data Quality, Metadata, Data Lineage, Azure DevOps, Git, Pull Requests, CI/CD, Root-Cause Analysis

#00354124