Data Engineer (Databricks, GCP), international retailer, remote

BrightStone Group; Amsterdam; Remote; EUR 70,000 - 110,000; Full time

Job summary

BrightStone Group in the Netherlands is seeking a Data Engineer to build and maintain batch, streaming, and near-real-time data pipelines on Databricks and GCP. You will write transformations in SQL and Python, ensure data quality, and help keep data secure and compliant.

The role includes mentoring colleagues and contributing to CI/CD and automation. Two seats are available: 1–3 years and 4–7 years of experience; remote from the Netherlands with visa-free work.

Qualifications

• Hands-on Databricks and Spark experience in production pipelines.
• Strong Python and advanced SQL skills for data transformations.
• Experience with data quality checks and reconciliation processes.

Responsibilities

• Build, test and run batch, streaming and near-real-time pipelines for sales, payments, orders and gift-card data.
• Write transformations and production data models in SQL, Python, BigQuery and dbt.
• Implement data-quality validation, monitoring and alerting.
• Ensure data safety through modelling and access controls.
• Contribute to CI/CD, infrastructure automation and documentation.
• Guide design decisions and mentor colleagues.

Skills

Python

SQL

Data pipelines

Data quality

Mentoring

CI/CD

Design decisions

Tools

Databricks

Spark

Delta Lake

Unity Catalog

Structured streaming

Workflows

BigQuery

dbt

GCP

Job description

Data Engineer (Databricks, GCP) · remote from the Netherlands · employed by BrightStone Group

A payment attempt, an order, a gift card, a sale in a store or online. At a large international retailer these start in different source systems, and this team turns them into one data product that analysts open in BigQuery without first having to find, combine and clean the data themselves. If a payment or an order is wrong in that product, everyone downstream is wrong with it.

There are two seats on this work, one for 1 to 3 years of experience and one for 4 to 7. The requirements read the same for both. I'm confirming with the client where they really differ, and you'll hear it on our call.

The work

• Build, test and run batch, streaming and near-real-time pipelines for sales, payment, order and gift-card data
• Write transformations and production data models in SQL, Python, BigQuery and dbt
• Reconciliation, data-quality checks, monitoring, and dead-letter queues for what fails
• Keep personal data safe through modelling and access controls
• CI/CD, code reviews, infrastructure automation, and documentation people can use
• Help guide design decisions and mentor colleagues

The team runs on GCP today. What they want you to bring is Databricks and lakehouse experience.

What you bring

• Hands-on Databricks: Spark, Delta Lake, Unity Catalog, structured streaming, Workflows
• Strong Python and advanced SQL
• Pipelines and data models you built and then kept running in production
• Data-quality validation, reconciliation and alerting
• Event-driven architecture, change data capture, schema evolution, and how to recover when something breaks
• Git, automated testing, CI/CD
• Working knowledge of IAM, PII handling, metadata and lineage

Good to have: BigQuery, Dataflow, Pub/Sub, Cloud Run, Cloud Storage. dbt, Apache Beam, Terraform, GitHub Actions. Sales, payments or other high-volume transactional data. Data mesh.

Before you apply

• You'd be employed by BrightStone Group and work for the retailer. It isn't a job at the retailer itself. I'll tell you who they are on our first call
• Employment only. No freelancers or subcontractors, because the client won't accept them
• You live in the Netherlands and can work here without visa sponsorship
• Travel is only for a team meeting, maybe once

#DataEngineer #Databricks #ApacheSpark #GCP #Netherlands