Data Engineer: Cloud Pipelines & Migrations

Netcompany Group A/S; Leeds; Hybrid; GBP 65,000 - 80,000; Full time

Benefits offered by this job

Hybrid work

Private medical

Pension

Life insurance

Certifications

Employee discounts

Job summary

Netcompany is seeking a data engineer to build the data foundations for a large-scale migration. You will design and operate repeatable data pipelines that move data from multiple sources into a single platform, owning data processing from ingestion to delivery.

You'll collaborate with the data lead, analysts, and platform teams, transforming extracts into consistent structures and implementing automated checks for completeness and accuracy.

Qualifications

• Hands-on experience building and operating data pipelines in a migration environment.
• Experience building pipelines on cloud platforms (Azure/AWS/GCP).
• Strong SQL and Python for data transformations and validation.
• Proficient with Spark or PySpark and Databricks or equivalent distributed processing.
• Experience with dbt for modular, tested transformations.
• Familiarity with Delta Lake transactional tables and batch loading.
• Infra as code using Terraform to provision cloud environments.

Responsibilities

• Design and operate repeatable data pipelines from ingestion to delivery.
• Collaborate with data lead, analysts, and platform teams across sources.
• Transform source extracts into consistent structures for migration to a unified platform.
• Develop maintainable ingestion and transformation code using SQL, Python, PySpark and dbt.
• Implement automated checks to ensure data completeness and accuracy.
• Monitor and diagnose failed or partial data processing and recover from issues.
• Maintain auditable records of data sources, mappings, and outcomes with proper controls.

Skills

Data pipelines

Cloud platforms

SQL

Python

Spark / PySpark

dbt

Delta Lake

Terraform

Data migration

Tools

Databricks

Azure

AWS

Google Cloud

Job description

Netcompany is seeking a data engineer to build the data foundations for a large-scale migration. You will design and operate repeatable data pipelines that move data from multiple sources into a single platform, owning data processing from ingestion to delivery.

You'll collaborate with the data lead, analysts, and platform teams, transforming extracts into consistent structures and implementing automated checks for completeness and accuracy.