Lead Data Engineer - dbt & Databricks Migration (Remote)

twentysix; Hilversum; On-site; EUR 90,000 - 120,000; Full time

Benefits offered by this job

Fully remote

Contract role

Job summary

Monks Technology Services seeks a Lead Data Engineer to drive a large-scale data platform modernization. You will guide migrating legacy Python/Spark ETL to dbt Core models on Databricks, modernizing production reporting for analytics and BI.

You will provide hands-on leadership, design scalable models, and ensure output parity with legacy pipelines, in a collaborative, CI/CD‑gated environment. Remote work is supported.

Qualifications

• 5+ years of production experience with dbt Core; dbt Certified Developer certification preferred.
• Hands-on experience leading data engineering teams, mentoring engineers, reviewing code, and owning delivery.
• Advanced knowledge of dbt, including Jinja macros, sources.yml, schema.yml tests, reusable model patterns, and incremental models.
• Hands-on experience with Databricks, Delta Lake, Unity Catalog, and Databricks SQL Warehouses.
• Advanced SQL skills, including CTEs, window functions, complex transformations, and Jinja templating.
• Experience migrating Python-based ETL pipelines into modular dbt models.
• Experience validating output parity between modernized models and legacy pipelines.
• Proficiency with Git-based PR workflows and CI/CD gates, such as GitHub Actions running dbt compile and dbt test.
• Strong technical leadership and stakeholder communication.

Responsibilities

• Lead the end-to-end migration of legacy Python/Spark ETL configurations into modular, maintainable dbt Core models.
• Provide hands-on technical leadership, mentorship, code review, and delivery guidance to data engineers.
• Partner with internal client engineers to define migration priorities, technical standards, and implementation plans.
• Reverse-engineer existing Python ETL logic and translate it into scalable dbt SQL models, staging layers, and reusable Jinja macros.
• Design foundational shared macros, dimension tables, and domain models.
• Implement dbt sources, incremental model patterns, and schema.yml tests (not_null, unique, relationships).
• Validate output parity between new models and legacy pipelines before go-live.
• Establish and uphold engineering standards for model design, documentation, testing, and maintainability.
• Manage delivery through a Git-based PR-reviewed workflow with CI/CD gates.
• Identify migration risks and communicate progress to stakeholders.
• Support scheduled dbt orchestration and production deployment within Databricks ecosystem.
• Collaborate across distributed teams.

Skills

dbt Core

Led data teams

Databricks

Delta Lake

Unity Catalog

Databricks SQL

Advanced SQL

GitHub Actions

CI/CD

Fivetran

Tools

dbt Core

Databricks

GitHub Actions

CI/CD pipelines

Fivetran

Job description

Monks Technology Services seeks a Lead Data Engineer to drive a large-scale data platform modernization. You will guide migrating legacy Python/Spark ETL to dbt Core models on Databricks, modernizing production reporting for analytics and BI.

You will provide hands-on leadership, design scalable models, and ensure output parity with legacy pipelines, in a collaborative, CI/CD‑gated environment. Remote work is supported.