Data Engineer- Databricks & Treasury Data Lake

Set Europa; Málaga; Remote; EUR 55,000 - 85,000; Full time

Job summary

SET Europa is seeking an experienced Data Engineer to join a fully remote project focused on Databricks and Treasury Data Lake environments. You will design, build, and optimize end-to-end data pipelines, ensure data quality, and collaborate with stakeholders to deliver scalable data solutions.

The role requires hands-on expertise with Databricks, DBT, SQL, and modern Lakehouse architectures, plus experience with CI/CD and data governance. Remote work from Spain is supported.

Qualifications

• Experience as a Data Engineer in enterprise data environments.
• Strong SQL and data transformation skills.
• Knowledge of data governance, automation, and CI/CD practices.

Responsibilities

• Design, build, and maintain end-to-end data ingestion pipelines.
• Develop and optimize transformations across lakehouse layers using Databricks and DBT.
• Implement data quality controls, monitoring, and alerting.
• Develop and maintain CI/CD pipelines with GitHub and Azure DevOps.
• Collaborate with stakeholders to translate requirements into robust data solutions.
• Support governance, documentation and release management.
• Deliver reliable data products for analytics and reporting.

Skills

Advanced SQL

CI/CD practices

Stakeholder collaboration

Tools

Databricks

DBT

GitHub

Azure DevOps

Lakeflow Connect

Job description

SET Europa provides near shore, recruitment and outsourcing services and solutions for immediate project needs.

Our client is a recruitment consultancy specialising in IT, Finance, HR, and Digital Technology. Our client is seeking an experienced Data Engineer specializing in Databricks and Treasury Data Lake environments to join a fully remote project.

The successful candidate will be responsible for designing, developing, maintaining, and optimizing end-to-end data pipelines within a modern Lakehouse architecture, ensuring reliable, scalable, and high-performance data solutions. This role requires strong hands‑on experience with Databricks, DBT, advanced SQL, and enterprise data integration, alongside a solid understanding of data governance, automation, and CI/CD practices.

RESPONSIBILITIES

• Design, build, and maintain end‑to‑end data ingestion pipelines integrating multiple source systems.
• Develop and optimise data transformations across bronze, silver and gold layers within a Databricks Lakehouse architecture.
• Implement automated data quality controls, monitoring frameworks and alerting mechanisms to ensure data integrity and reliability.
• Design and maintain data models and transformation logic using DBT and advanced SQL.
• Leverage Databricks Lakehouse Federation and Lakeflow Connect to support efficient data integration and accessibility.
• Develop and maintain CI/CD pipelines using GitHub and Azure DevOps, ensuring controlled deployments, testing and version management.
• Identify and resolve performance bottlenecks, improving scalability, maintainability and efficiency.
• Collaborate with business and technical stakeholders to translate requirements into robust data engineering solutions.
• Support data governance standards, documentation, troubleshooting and release management activities.
• Contribute to analytics and reporting capabilities by delivering reliable, high‑quality data products.

Requirements

• Proven professional experience as a Data Engineer in enterprise data environments.
• Strong hands‑on expertise with Databricks and modern Data Lake/Lakehouse architecture.
• Practical experience developing data transformations using DBT (Data Build Tool).
• Advanced SQL skills, including query optimisation and complex data transformations.
• Experience designing and managing data pipelines across bronze, silver and gold layers.
• Knowledge of Databricks Lakehouse Federation and Lakeflow Connect.
• Experience implementing data quality frameworks, monitoring and automated alerting.