Databricks Data Engineer - Real-Time Pipelines & Lakehouse

Deloitte Netherlands; Amsterdam; On-site; EUR 65,000 - 90,000; Full time

Job summary

Deloitte Netherlands is seeking a Databricks Data Engineer to design, build, and optimize scalable data pipelines using Databricks, PySpark, and Spark SQL in Amsterdam. You will implement medallion architecture, Delta Lake, and CI/CD for notebooks, jobs, and pipelines, collaborating with cloud platform engineers to ensure secure, cost-efficient solutions.

Ideal candidates hold a technical Master's degree, 4+ years in data engineering (consulting a plus), and strong English and Dutch language

Qualifications

• Strong hands-on experience with Databricks and Apache Spark (PySpark and Spark SQL)
• Deep understanding of Delta Lake and medallion architecture
• Proficiency in SQL and one or more data engineering related programming languages (Python, Scala)
• Relevant Databricks certifications (Data Engineer Associate/Professional, Spark Developer Associate, Data Analyst Associate)
• Excellent command of English and Dutch.

Responsibilities

• Develop, optimize, and operate ETL/ELT pipelines in Databricks using PySpark, SparkSQL, and Scala.
• Run migration programmes to Databricks, including legacy ETL processes into scalable data pipelines.
• Build robust data pipelines on Databricks for real time and batch processing.
• Configure and manage Databricks clusters and jobs for cost and performance efficiency.
• Apply CI/CD to automate testing and deployment of notebooks, jobs, and pipelines.
• Drive adoption of Databricks Lakehouse architecture and data engineering practice across our clients.

Job description

Deloitte Netherlands is seeking a Databricks Data Engineer to design, build, and optimize scalable data pipelines using Databricks, PySpark, and Spark SQL in Amsterdam. You will implement medallion architecture, Delta Lake, and CI/CD for notebooks, jobs, and pipelines, collaborating with cloud platform engineers to ensure secure, cost-efficient solutions.

Ideal candidates hold a technical Master's degree, 4+ years in data engineering (consulting a plus), and strong English and Dutch language