Senior Data Engineer – Cloud Data Pipelines & Migration

Netcompany; Leeds; On-site; GBP 60,000 - 85,000; Full time

Benefits offered by this job

Hybrid working model with some flexi-

Private Medical Health care via Vital

Pension contribution, Life Assurance

Professional certifications supported

Retail discounts across lifestyle

Employee Resource Groups

Job summary

Netcompany in Leeds is seeking a Data Engineer to build the data foundations for a large-scale migration. You will design and operate repeatable pipelines that move varied source data into a single platform, owning the data processing from ingestion to delivery.

You will work with the migration team, data lead, analysts and platform teams, writing modular, tested transformations in SQL, Python and PySpark, using dbt and cloud services.

Qualifications

• Hands-on experience building and operating data pipelines in a migration environment.
• Experience with Spark, PySpark, and Databricks for distributed processing.
• Strong SQL and Python for transformations and validation.
• Experience with cloud platforms (Azure, AWS, GCP) and data lakehouse tech like Delta Lake.
• Experience with Terraform or IaC to provision cloud environments.

Responsibilities

• Design and operate repeatable pipelines to move varied data into a single platform.
• Collaborate with data lead, analysts and platform team throughout delivery.
• Build and maintain ingestion and transformation code.
• Implement automated checks for data completeness and accuracy.
• Monitor and recover failed data processing and maintain auditable data lineage.

Skills

Data pipelines

Azure

AWS

Google Cloud

Spark/PySpark

dbt

SQL

Python

Orchestration tools

Tools

Databricks

Terraform

Job description

Netcompany in Leeds is seeking a Data Engineer to build the data foundations for a large-scale migration. You will design and operate repeatable pipelines that move varied source data into a single platform, owning the data processing from ingestion to delivery.

You will work with the migration team, data lead, analysts and platform teams, writing modular, tested transformations in SQL, Python and PySpark, using dbt and cloud services.