Data Engineer - Databricks

UMATR; Amsterdam; On-site; EUR 96,000 - 138,000; Full time

Job summary

UMATR is collaborating with a European technology company undergoing a major data platform transformation and is seeking a senior Databricks Data Engineer for an initial 6-month freelance contract.

You'll join an environment where the data platform is being actively built and modernised, with Databricks at the centre of the architecture. You will design scalable pipelines and optimise workloads while collaborating across teams.

Qualifications

• Minimum 5+ years in Data Engineering.
• Hands-on Databricks experience.
• Strong Delta Lake and Lakehouse understanding.
• Experience building and optimizing production data pipelines.
• Experience with major cloud providers.

Responsibilities

• Design scalable data pipelines within Databricks.
• Develop production-grade workloads using PySpark, Python and SQL.
• Work with Delta Lake and Lakehouse architecture.
• Support migration of workloads into Databricks.
• Optimize Spark workloads and performance.
• Collaborate with Data Scientists and Cloud Engineers.
• Implement data quality, testing and monitoring.
• Contribute to the wider architecture and technical direction of the platform.

Skills

Data Engineering

Databricks

PySpark

Python

SQL

Delta Lake

Lakehouse

Production pipelines

Cloud platforms

Independent work

Tools

Databricks

Delta Lake

Spark

Azure

AWS

GCP

Job description

We’re working with a European technology company undergoing a major data platform transformation and they’re looking for a senior Databricks Data Engineer to join the team on an initial 6-month freelance contract.

You’ll be coming into an environment where the data platform is being actively built and modernised, with Databricks at the centre of the architecture.

What you'll be working on

• Designing and building scalable data pipelines within Databricks
• Developing production-grade workloads using PySpark, Python and SQL
• Working extensively with Delta Lake and Lakehouse architecture
• Supporting the migration of existing workloads into Databricks
• Optimising Spark workloads and improving pipeline performance
• Working with Data Scientists, Analytics Engineers and Cloud Engineers
• Implementing data quality, testing and monitoring
• Contributing to the wider architecture and technical direction of the platform

We're looking for someone who has

• 5+ years of experience in Data Engineering
• Strong hands-on Databricks experience
• Strong understanding of Delta Lake and Lakehouse architecture
• Experience building and optimising production data pipelines
• Experience working with Azure, AWS or GCP
• Experience working within large-scale data environments
• The ability to operate independently and hit the ground running