Data Engineer: Databricks, PySpark & Delta Lake

Kyndryl Inc.; Noord-Holland; Hybrid; EUR 70,000 - 110,000; Full time

Benefits offered by this job

Be Well program

Cutting-edge learning opportunities

Job summary

Kyndryl is seeking a Data Engineer to design and maintain scalable data pipelines on a cloud-based platform. You will work with Databricks, PySpark, Python, and SQL to ingest, transform, and orchestrate data across diverse sources, implementing Delta Lake architectures and robust data quality checks.

In a hybrid environment, you will optimize workloads, support CI/CD, and collaborate with teams to ensure reliable data services.

Qualifications

• Must Have: Databricks, PySpark, Python, SQL, ETL/ELT, Delta Lake, data pipeline development.
• Preferred: Azure Data Factory (ADF), ADLS Gen2, Azure Databricks, Airflow, Unity Catalog, CI/CD, Git, Terraform and data-quality frameworks.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Develop data-processing workloads using Databricks, PySpark, Python, and SQL.
• Build ingestion and transformation pipelines for structured and semi-structured data.
• Implement Delta Lake and Medallion Architecture (Bronze/Silver/Gold) patterns.
• Perform data transformation, validation, reconciliation, and data-quality checks.
• Optimize Databricks jobs, Spark workloads, and SQL queries for performance.
• Integrate data from multiple source systems into enterprise data platforms.
• Develop and maintain pipeline orchestration and scheduling.
• Implement monitoring, logging, exception handling, and pipeline recovery mechanisms.
• Support CI/CD and automated deployment of data-engineering workloads.

Skills

Databricks

PySpark

Python

SQL

ETL/ELT

Delta Lake

data pipeline development

Tools

Azure Data Factory

ADLS Gen2

Azure Databricks

Airflow

Unity Catalog

CI/CD

Git

Terraform

Job description

Kyndryl is seeking a Data Engineer to design and maintain scalable data pipelines on a cloud-based platform. You will work with Databricks, PySpark, Python, and SQL to ingest, transform, and orchestrate data across diverse sources, implementing Delta Lake architectures and robust data quality checks.

In a hybrid environment, you will optimize workloads, support CI/CD, and collaborate with teams to ensure reliable data services.