Remote Data Engineer (PySpark, Fabric) – Master Data

Proactive Appointments; Exeter; Hybrid; GBP 74,000 - 83,000; Full time

Job summary

Proactive Appointments seeks a Data Engineer to design and implement data pipelines using Microsoft Fabric and PySpark, cleansing and standardising customer data to build accurate master records across multiple systems. You will optimize matching models, ensure data quality, and document decisions for non-technical stakeholders.

The role requires strong Python (pandas, PySpark) and SQL skills, with experience in data cleaning and deduplication, and knowledge of GDPR practices.

Qualifications

• Experience building data pipelines in a cloud data platform, ideally Microsoft Fabric.
• Strong Python, including pandas and PySpark.
• Strong SQL, including Spark SQL or T-SQL.

Responsibilities

• Develop data matching pipelines, cleansing and standardising customer data.
• Tune matching models and build accurate master customer records across systems.
• Explain technical decisions and trade-offs clearly to non-technical colleagues.

Skills

Python

PySpark

SQL

Data Pipelines

Data Quality

Data Matching

Git

GDPR

Tools

Microsoft Fabric

Pandas

Spark SQL

T-SQL

Job description

Proactive Appointments seeks a Data Engineer to design and implement data pipelines using Microsoft Fabric and PySpark, cleansing and standardising customer data to build accurate master records across multiple systems. You will optimize matching models, ensure data quality, and document decisions for non-technical stakeholders.

The role requires strong Python (pandas, PySpark) and SQL skills, with experience in data cleaning and deduplication, and knowledge of GDPR practices.