Senior Data Engineer: Cloud Pipelines & Migration Expert

Netcompany1; Greater London; Hybrid; GBP 65,000 - 90,000; Full time

Benefits offered by this job

Hybrid working model

25 days' holiday

Private Medical Health care via Vital1

Pension contribution

Life Assurance

Professional certifications supported

Retail discounts

Employee Resource Groups

Job summary

Netcompany is seeking an experienced data engineer to build and operate scalable data pipelines for a large migration project. You will design repeatable ingestion, validation and transformation processes across cloud platforms, using SQL, Python, PySpark and dbt.

You will partner with data leads, analysts and platform teams to deliver reliable data processing, ensuring traceability, governance and auditable records throughout the migration lifecycle.

Qualifications

• Hands-on experience building and operating data pipelines in cloud environments.
• Experience with data validation, reconciliation queries and auditing.
• Proficiency in SQL and Python for data transformations and troubleshooting.

Responsibilities

• Build repeatable data ingestion and transformation processes from multiple sources.
• Transform source extracts into consistent structures for migration to a single platform.
• Develop maintainable ingestion and transformation code using SQL, Python, PySpark and dbt.

Skills

SQL

Python

PySpark

dbt

Delta Lake

Terraform

Cloud platforms

Orchestration tools

Tools

Databricks

Azure

AWS

Google Cloud

Job description

Netcompany is seeking an experienced data engineer to build and operate scalable data pipelines for a large migration project. You will design repeatable ingestion, validation and transformation processes across cloud platforms, using SQL, Python, PySpark and dbt.

You will partner with data leads, analysts and platform teams to deliver reliable data processing, ensuring traceability, governance and auditable records throughout the migration lifecycle.