Hybrid Azure Databricks Data Engineer in Lisbon

iTRTech Group; Lisboa; Hybrid; EUR 30,000 - 33,000; Full time

Benefits offered by this job

Hybrid work in Lisbon

Job summary

iTRTech Group in Lisbon is seeking an Azure Databricks Data Engineer to design, build, and maintain enterprise data pipelines using Azure Databricks, dbt and Airflow. The role covers data ingestion, transformation, modelling, orchestration and production troubleshooting.

The ideal candidate has 3+ years of experience and strong SQL/Python skills, with production exposure to Databricks, Delta Lake and Medallion Architecture.

Qualifications

• - 3+ years of Data Engineering experience in enterprise environments.
• - Advanced SQL and strong Python skills.
• - Production experience with Azure Databricks, ADLS Gen2, Delta Lake and Medallion Architecture.
• - Strong dbt experience, including models, incremental materialisations, tests, dependencies and troubleshooting.
• - Experience with Airflow/Astronomer, including DAGs, scheduling, retries and monitoring.
• - Data ingestion experience with APIs, SFTP/files and JDBC.
• - Knowledge of data modelling, SCD Type 2 and data lineage.
• - English and Portuguese at B2 level.
• - Availability for hybrid work in Lisbon.

Responsibilities

• Develop data pipelines using Azure Databricks, ADLS Gen2 and Delta Lake.
• Implement Medallion Architecture and transformation models with dbt.
• Create and maintain Airflow/Astronomer DAGs.
• Ingest data through APIs, SFTP/files and JDBC.
• Develop solutions using SQL, Python, Scala, Spark and PySpark.
• Implement data modelling, quality, metadata and lineage practices.
• Manage automated validations and deployments through Azure DevOps and CI/CD.
• Troubleshoot production issues and perform root-cause analysis.
• Produce technical documentation and communicate with stakeholders.

Skills

Advanced SQL

Python

Airflow

dbt

Azure Databricks

Delta Lake

ADLS Gen2

Git / CI/CD

Spark/PySpark

Tools

Azure DevOps

Airflow/Astronomer

dbt Analytics Engineering Certification

Microsoft Purview

Azure Monitor

Power BI

Delta Tables

Job description

iTRTech Group in Lisbon is seeking an Azure Databricks Data Engineer to design, build, and maintain enterprise data pipelines using Azure Databricks, dbt and Airflow. The role covers data ingestion, transformation, modelling, orchestration and production troubleshooting.

The ideal candidate has 3+ years of experience and strong SQL/Python skills, with production exposure to Databricks, Delta Lake and Medallion Architecture.