Hibrido BCN - Senior Data Engineer

Capitole Consulting; Barcelona; On-site; EUR 70,000 - 110,000; Full time

Benefits offered by this job

Professional development stipend

Private health insurance

Meal, transport, childcare vouchers

Wellhub subscription

Capitole Club discounts

Job summary

Capitole Consulting in Barcelona is seeking a Senior Data Engineer to drive the evolution of our Azure-based Data Platform and Lakehouse. You will build pipelines in ADF, design DAGs in Airflow, and craft Databricks transformations using PySpark and SQL.

You will collaborate with POs, architects, and Run/Ops teams to ensure scalable, observable data products, with a focus on Medallion architecture and solid data modeling. This is a dynamic, growth-focused role in a fast-moving team.

Qualifications

• 6+ years of experience in data engineering.
• Strong technical expertise with Python (PySpark) and SQL.
• Hands-on experience with Azure-based Data Platform including Databricks, Unity Catalog, Airflow, and Data Factory.
• Knowledge of Data Warehouse, Data Lake and Lakehouse architectures and Medallion architecture.
• Understanding of partitioning, optimization, and performance for batch and streaming pipelines.

Responsibilities

• Implement scalable data pipelines using Azure Data Factory and Apache Airflow.
• Develop data transformations in Azure Databricks (PySpark/SQL).
• Create and extend data products following Medallion Architecture (Bronze, Silver, Gold).
• Optimize performance, cost and reliability of data workloads.
• Lead architecture reviews and promote engineering excellence.

Skills

Python (PySpark)

SQL

Data engineering

Tools

Databricks

Azure Data Factory

Apache Airflow

Unity Catalog

Power BI

Tableau

Job description

Capitole is growing fast, and we want you to be part of the journey!

We are looking for a Senior Data Engineer with +6 years of experience to help drive the evolution of our Azure-based Data Platform and Lakehouse architecture.

This role is focused on Data Products development. The successful candidate will build pipelines in ADF, DAGs in Airflow and work closely with POs to refine business requirements and transform business needs into reliable Data Products in Databricks.

Responsibilities:

• Implement scalable, reliable and reusable data pipelines using Azure Data Factory and Apache Airflow.
• Develop data transformations in Azure Databricks (PySpark / SQL) to create/extend data products following the Medallion Architecture layers:
  • Bronze (raw ingestion)
  • Silver (cleansed & conformed)
  • Gold (business-ready data products)
• Optimize performance, cost, and reliability of data workloads.
• Raise important questions during business maturation in order to detect risks before development stage.

Technical Leadership

• Participate in architecture reviews and technical decision-making.
• Promote engineering excellence through code reviews and design reviews.
• Collaborate with Architects, Product Owners in data product technical maturation.

Support & Operations (Run Team Collaboration)

• Provide support to the Run/Operations team for production pipelines and data products for complex topics.
• Troubleshoot and resolve data incidents, failures, and performance issues.
• Participate in incident management, root cause analysis, and post-mortems.
• Ensure proper handover, documentation, and operational readiness of developed solutions.
• Contribute to improving runbooks, monitoring, and operational processes.

Requirements:

Technical Skills

• 6+ years of experience in data engineering.
• Strong technical expertise with:
  • Python (PySpark)
  • SQL (advanced level)
• Hands-on experience with Azure-based Data Platform including:
  • Databricks
  • Unity Catalog
  • Apache Airflow
  • Azure Data Factory
  • Reporting and BI tools (Power BI, Tableau or similar)
• Solid understanding of data modelling fundamentals, main architectures and design patterns:
  • Data Warehouse, Data Lake and Data Lakehouse architectures
  • Medallion architecture
  • Metadata-driven architecture
  • ETL/ELT design patterns
• Understanding of:
  • Partitioning, optimization, and performance
  • Batch and streaming pipelines

Soft Skills

• Strong problem-solving and analytical thinking.
• Ability to work in cross-functional teams.
• Ownership mindset and attention to detail.
• Good communication skills with both technical and business stakeholders.

What we’re looking for in you:

• Built metadata-driven orchestration frameworks.
• Designed reusable ETL frameworks.
• Led an Airflow implementation or migration.
• Implemented observability and data quality platforms.
• Acted as a lead engineer or senior individual contributor in a modern Lakehouse environment.

Why you'll love working with us

• €1,200/year for your professional development (trainings, events, certifications…)
• Monthly feedback sessions with your team to keep learning and growing
• Private health insurance fully covered by Capitole
• Payflow benefits (meal, transport, childcare vouchers)
• Wellhub subscription
• Discounts with top brands through Capitole Club

Life at Capitole

• Team buildings every two months (our summer party and Christmas dinner are legendary!)
• Football, padel, and basketball teams sponsored by Capitole
• Tech communities where sharing knowledge is part of our DNA
• And above all… an AMAZING TEAM that you’ll love being a part of!

Want to learn more about us? capitole-consulting.com

See what our teams are saying about us: Glassdoor Reviews

The employee will adhere to information security policies:

• Will have access to confidential information related to Capitole and the project they are working on.
• Must comply with the security and internal policies of both the company and the client.
• Must sign an NDA (Non-Disclosure Agreement).