Data Engineer - Databricks & Lakehouse Expert

Avanade Inc.; Madrid; On-site; EUR 50,000 - 70,000; Full time

Job summary

Avanade Inc. is seeking a Data Engineer in Madrid to design and build scalable data pipelines using Databricks and cloud technologies. You will transform data with Python, PySpark, and SQL, while working with Delta Lake and lakehouse architectures to handle large volumes of diverse data.

Join a global team, collaborate with architects, scientists, and BI experts, and contribute to data quality, governance, and security in international projects. Fluent English preferred.

Qualifications

• Experience designing scalable data pipelines and ETL/ELT processes.
• Proficient in Python, SQL and PySpark development.
• Hands-on with Databricks and Delta Lake in cloud environments.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes.
• Build modern data solutions using Databricks and cloud technologies.
• Develop and optimize data transformations using Python, PySpark and SQL.
• Work with Delta Lake and lakehouse architectures to process large volumes of data.
• Implement data ingestion, transformation, integration, and quality processes.
• Collaborate with Data Architects, Data Scientists, BI specialists, and business stakeholders.
• Apply best practices around data quality, performance, governance, security, and maintainability.
• Participate in international projects, communicating with clients and colleagues in English.

Skills

Python

PySpark

SQL

Data pipelines

ETL/ELT

Databricks

Cloud technologies

Tools

Databricks

Delta Lake

Lakehouse architectures

Job description

Avanade Inc. is seeking a Data Engineer in Madrid to design and build scalable data pipelines using Databricks and cloud technologies. You will transform data with Python, PySpark, and SQL, while working with Delta Lake and lakehouse architectures to handle large volumes of diverse data.

Join a global team, collaborate with architects, scientists, and BI experts, and contribute to data quality, governance, and security in international projects. Fluent English preferred.