Senior Data Engineer — Cloud Data Pipelines Lead

Netcompany

Ledston

Hybrid

GBP 70,000 - 100,000

Full time

6 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Hybrid working model
25 days holiday
Private Medical Health care via Vitaly
Pension contribution
Life Assurance
Professional certifications supported
Retail discounts
Employee Resource Groups

Job summary

Netcompany is seeking a data engineer to build and operate the data foundations for a large-scale migration. You’ll design repeatable pipelines from ingestion to delivery, collaborating with cross-functional teams to ensure data quality and timely delivery.

You’ll work with SQL, Python, PySpark and dbt on cloud platforms, implementing automated checks and monitoring. A strong emphasis on secure data handling and auditable records is key.

Qualifications

  • Hands-on experience building and operating data pipelines in migration environments.
  • Experience building pipelines on Azure, AWS or Google Cloud.
  • Strong SQL and Python skills for transformations, validation and reconciliation queries.
  • Experience with Spark, PySpark, Databricks or comparable distributed processing technologies.
  • Experience building modular, tested and documented transformations using dbt or an equivalent approach.
  • Experience scheduling, monitoring and recovering pipelines using cloud-native or equivalent orchestration tools.
  • Experience with Delta Lake, including transactional tables, schema changes and repeatable batch loading.
  • Experience using Terraform or equivalent infrastructure as code to provision and maintain consistent cloud environments.

Responsibilities

  • Work as part of a migration team collaborating with the data lead, analysts, the platform team and source organisations throughout the delivery.
  • Build repeatable processes to receive, unpack, validate and ingest data from multiple sources.
  • Transform source extracts into consistent structures for migration to a single platform.
  • Develop maintainable ingestion and transformation code using SQL, Python, PySpark and dbt.
  • Implement automated checks to ensure completeness and accuracy throughout the migration.
  • Monitor, diagnose and recover failed or incomplete data processing.
  • Maintain auditable records of data sources, mappings, transformations and outcomes, with secure data handling controls.

Skills

Data pipelines
SQL
Python
Orchestration
Data validation
Cloud data processing

Tools

Spark
PySpark
Databricks
Delta Lake
Terraform

Job description

Netcompany is seeking a data engineer to build and operate the data foundations for a large-scale migration. You’ll design repeatable pipelines from ingestion to delivery, collaborating with cross-functional teams to ensure data quality and timely delivery.

You’ll work with SQL, Python, PySpark and dbt on cloud platforms, implementing automated checks and monitoring. A strong emphasis on secure data handling and auditable records is key.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer: Cloud Pipelines & Migration Expert
Senior Data Engineer: Cloud Pipelines & Migration Expert

Netcompany1 • Greater London

Hybrid
GBP 65,000 - 90,000
Hybrid working model
25 days' holiday
Private Medical Health care via Vital1
+5
Senior Data Engineer: Cloud Data Pipelines & Migration
Senior Data Engineer: Cloud Data Pipelines & Migration

Netcompany1 • Leeds

Hybrid
GBP 65,000 - 90,000
Hybrid working model
25 days holiday
Private Medical Health care via Vitaly
+5
Hybrid Data Engineer—Cloud Pipelines & Migrations
Hybrid Data Engineer—Cloud Pipelines & Migrations

Netcompany1 • Leeds

Hybrid
GBP 60,000 - 90,000
Hybrid working model
25 days holiday
Private medical health care
+5
Senior Data Engineer: Cloud Data Pipelines & Migrations
Senior Data Engineer: Cloud Data Pipelines & Migrations

Netcompany Group A/S • Leeds

Hybrid
GBP 60,000 - 90,000
Hybrid working model
Private Medical Health care via Vitaly
Pension contribution and Life Assuar
+3
Data Engineer – Cloud Pipelines & Migration
Data Engineer – Cloud Pipelines & Migration

Netcompany • Ledston

Hybrid
GBP 70,000 - 95,000
Hybrid working
25 days holiday
Private Medical Health care
+4
Data Engineer: Cloud Pipelines & Migrations
Data Engineer: Cloud Pipelines & Migrations

Netcompany Group A/S • Leeds

Hybrid
GBP 65,000 - 80,000
Hybrid work
Private medical
Pension
+3
Senior Data Engineer – Cloud Data Pipelines & Migration
Senior Data Engineer – Cloud Data Pipelines & Migration

Netcompany • Leeds

On-site
GBP 60,000 - 85,000
Hybrid working model with some flexi-
Private Medical Health care via Vital
Pension contribution, Life Assurance
+3
Senior Data Engineer: Migration & Data Pipelines (Hybrid)
Senior Data Engineer: Migration & Data Pipelines (Hybrid)

Netcompany • Greater London

Hybrid
GBP 75,000 - 110,000
Hybrid working model
Private medical health care
Pension contribution
+3
Hybrid Cloud Data Pipeline Engineer for Migrations
Hybrid Cloud Data Pipeline Engineer for Migrations

Netcompany • Leeds

Hybrid
GBP 60,000 - 90,000
Hybrid working model with some flexi
Private Medical Health care via Vitaly
Pension contribution
+4
Senior Data Engineer - Cloud Pipelines & Analytics
Senior Data Engineer - Cloud Pipelines & Analytics

Data Idols • Greater London

Hybrid
GBP 80,000 - 95,000