Lead Data Engineer- Databricks

Marktplaats; Amsterdam; On-site; EUR 85,000 - 120,000; Full time

Benefits offered by this job

Base salary guidance

Annual bonus

Employee Assistance Program

Collaborative culture

Job summary

Marktplaats is seeking a Data Engineer for the Data Platform team to build and operate a data/ML platform. You will own the self-serve data platform, manage data collection and lake, and deliver high-quality data products for analytics and ML workloads.

You’ll work on batch and streaming pipelines, orchestration, and APIs, ensuring reliability, scalability and governance in a cloud-first environment.

Qualifications

• Experience with Databricks and cloud-native data apps.
• Building real-time and batch data pipelines on AWS.
• Strong data engineering background to support teammates.
• Hands-on Spark, Python and PySpark experience.
• AWS data management, governance, cost optimization.
• Ensure data quality, schema governance, monitoring.
• Experience with Airflow and Databricks workflows.
• Containerization with Docker and Kubernetes.
• Proficiency with Terraform/Terragrunt.
• Data validation and SQL fundamentals.
• Collaborative, autonomous team player with clear comms.
• Excellent English communication.

Responsibilities

• Independently develop and deliver high-quality features for the Data/ML Platform.
• Refactor and translate data products and complete tasks to high standards.
• Own self-serve data platform including data collection and lake management.
• Build and maintain batch and streaming data pipelines, storage and APIs.
• Collaborate with data engineers, back end developers and data scientists.
• Ensure platform reliability, scalability and performance.

Skills

Python

SQL

Data governance

Data quality

English communication

Team collaboration

AWS

Databricks

Spark

Airflow

Kafka

Docker

Kubernetes

Terraform

Terragrunt

Tools

Databricks

AWS

Spark

Kafka

Airflow

Docker

Kubernetes

Delta Lake

Parquet

Terraform

Terragrunt

MLflow

Job description

What You’ll Do & Who You Are

In the Marktplaats data and analytics teams, data is at the heart of everything we do. As a Data Engineer of the Data Platform team at Marktplaats you will be relied on to independently develop and deliver high-quality features for our new Data/ML Platform, refactor and translate our data products and finish various tasks to a high standard. You will be the cornerstone of the platform’s reliability, scalability and performance, working hands on with batch and streaming data pipelines, storage solutions and APIs that serve complex analytical and ML workloads. The role encompasses ownership of the self-serve data platform, including data collection, lake management, orchestration, processing, and distribution.

What We’re Looking For

Success in this role requires a high level of technical proficiency in modern data technologies and cloud-based environments, Databricks, AWS, Spark, Python, Kafka, Airflow, and deep expertise in data management and governance frameworks.

• Experience with Databricks (Lakehouse, ML/MosaicAI, Unity Catalog, MLflow, Mosaic AI, model serving etc).
• Proven experience on building cloud native data intensive applications (both real time and batch based). AWS experience is preferred.
• Strong background in Data Engineering to support other Data Engineers, Back Enders and Data Scientists in building data products and services.
• Hands‑on experience of building and maintaining Spark applications. Python and PySpark(Scala Spark is a plus)
• Experienced in AWS Cloud usage and data management (automation, data governance, cost optimisation, delivering reliable & scalable data solutions)
• Ensure data quality, schema governance and monitoring across pipelines.
• Experience with orchestrators such as Airflow, Databricks workflows.
• Solid experience with containerization and orchestration technologies (e.g., Docker, Kubernetes).
• Fundamental understanding of various Parquet, Delta Lake and other OTFs file formats.
• Proficiency on an IaC tool such as Terraform or Terragrunt.
• Data validation/analysis skills & proficiency in SQL is considered as a foundational skill.
• Collaborate in a small, fast moving team with high levels of autonomy and impact.
• Strong written and verbal English communication skill and proficient in communicating with non-technical stakeholders.

Benefits

Life at Marktplaats comes with its perks! Our colleagues enjoy the following benefits:

• Attractive Base Salary
• Participation in our Short-Term Incentive Plan (Annual Bonus)
• Employee Assistance Program: 24/7 support for you and your family
• Collaborative Culture: “Win together, lose together” is one of our key behaviours. You’ll be part of a supportive, growth-minded environment.

Equal Opportunity Statement

Marktplaats is an equal opportunity employer, and we value diversity. We do not discriminate on the basis of race, religion, colour, national origin, gender, sexual orientation, age, marital status, or disability status.