Databricks Data Engineer — Hybrid (Barcelona)

TechYard; Barcelona; Hybrid; EUR 70,000 - 110,000; Full time

Job summary

TechYard in Barcelona is seeking an experienced Databricks Data Engineer to design, build, and optimise scalable data pipelines and data platforms using Databricks, Apache Spark, and cloud technologies. You will collaborate with data architects, analysts, data scientists, and business stakeholders to transform complex datasets into reliable, high-quality data solutions that drive business value.

The role demands strong Python, PySpark, SQL skills and experience with Delta Lake and lakehouse

Qualifications

• Proven experience as a Data Engineer with Databricks.
• Strong proficiency in Python, PySpark, and SQL.
• Hands-on experience with Apache Spark and Delta Lake.
• Experience designing and implementing ETL/ELT pipelines.
• Knowledge of cloud data platforms (Azure/AWS/GCP).
• Experience with data modelling, warehousing, and lakehouse architectures.
• Familiarity with Git, CI/CD, and data orchestration.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and Apache Spark.
• Build and optimise data processing workflows using PySpark and SQL.
• Develop and manage data lakehouse architectures using Delta Lake and the Databricks Lakehouse Platform.
• Integrate data from multiple sources, ensuring data quality, consistency, and reliability.
• Implement data transformation, orchestration, and workflow automation.
• Optimise pipeline performance, scalability and cost efficiency.
• Apply data governance, security, and access control best practices.
• Collaborate with cross-functional teams to deliver robust, production-ready data solutions.
• Monitor, troubleshoot, and improve existing data pipelines and data platforms.

Skills

Python

PySpark

SQL

Databricks

Apache Spark

ETL/ELT

Data governance

Agile

Tools

Delta Lake

Azure

AWS

Google Cloud

Git

CI/CD

Terraform

Lakehouse Platform

Job description

TechYard in Barcelona is seeking an experienced Databricks Data Engineer to design, build, and optimise scalable data pipelines and data platforms using Databricks, Apache Spark, and cloud technologies. You will collaborate with data architects, analysts, data scientists, and business stakeholders to transform complex datasets into reliable, high-quality data solutions that drive business value.

The role demands strong Python, PySpark, SQL skills and experience with Delta Lake and lakehouse