Databricks Data Engineer - Cloud Pipelines & Lakehouse

Avanade Inc.; Madrid; On-site; EUR 60,000 - 75,000; Full time

Job summary

Avanade Inc. in Madrid is seeking a Data Engineer to accelerate data-driven decisions for clients using Microsoft cloud technologies.

You will design scalable data pipelines, implement ETL/ELT processes, and work with Databricks, Python, PySpark, SQL and Delta Lake in lakehouse architectures. Join a global team collaborating with Data Architects, Data Scientists, BI specialists, and stakeholders while communicating in English on international projects.

Qualifications

• Professional experience as a Data Engineer or similar.
• Hands-on Databricks in production environments.
• Strong knowledge of Python / PySpark and SQL.
• Experience designing and implementing ETL/ELT pipelines.
• Knowledge of Delta Lake, Lakehouse architectures, and data modelling.
• Experience with cloud data platforms, ideally Microsoft Azure.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes.
• Build modern data solutions using Databricks and cloud technologies.
• Develop and optimize data transformations using Python, PySpark and SQL.
• Work with Delta Lake and lakehouse architectures to process large volumes of structured and unstructured data.
• Implement data ingestion, transformation, integration, and quality processes.
• Collaborate with Data Architects, Data Scientists, BI specialists, and business stakeholders.
• Apply best practices around data quality, performance, governance, security, and maintainability.
• Participate in international projects, communicating with clients and colleagues in English.

Skills

Databricks

Python

PySpark

SQL

ETL/ELT pipelines

Delta Lake

Lakehouse architectures

CI/CD

Git

English communication

Tools

Azure Data Factory

Azure Data Lake Storage

Synapse Analytics

Microsoft Fabric

Job description

Avanade Inc. in Madrid is seeking a Data Engineer to accelerate data-driven decisions for clients using Microsoft cloud technologies.

You will design scalable data pipelines, implement ETL/ELT processes, and work with Databricks, Python, PySpark, SQL and Delta Lake in lakehouse architectures. Join a global team collaborating with Data Architects, Data Scientists, BI specialists, and stakeholders while communicating in English on international projects.