Data Engineer

Proactive Appointments; Exeter; Hybrid; GBP 74,000 - 83,000; Full time

Job summary

Proactive Appointments seeks a Data Engineer to design and implement data pipelines using Microsoft Fabric and PySpark, cleansing and standardising customer data to build accurate master records across multiple systems. You will optimize matching models, ensure data quality, and document decisions for non-technical stakeholders.

The role requires strong Python (pandas, PySpark) and SQL skills, with experience in data cleaning and deduplication, and knowledge of GDPR practices.

Qualifications

• Experience building data pipelines in a cloud data platform, ideally Microsoft Fabric.
• Strong Python, including pandas and PySpark.
• Strong SQL, including Spark SQL or T-SQL.

Responsibilities

• Develop data matching pipelines, cleansing and standardising customer data.
• Tune matching models and build accurate master customer records across systems.
• Explain technical decisions and trade-offs clearly to non-technical colleagues.

Skills

Python

PySpark

SQL

Data Pipelines

Data Quality

Data Matching

Git

GDPR

Tools

Microsoft Fabric

Pandas

Spark SQL

T-SQL

Job description

Data Engineer

Contract - Outside IR35 | £400-£450 per day | 6 Weeks Inital Contract Length | Remote (Inital 2-3 days on-site to kick start the project)

Our client is looking for an experienced Data Engineer with strong Microsoft Fabric, PySpark and Splink experience. You'll be responsible for developing data matching pipelines, cleansing and standardising customer data, tuning matching models and building accurate master customer records across multiple systems.

• Experience building and supporting data pipelines in a cloud data platform, ideally Microsoft Fabric.


• Strong Python, including pandas and PySpark.


• Strong SQL, including Spark SQL or T-SQL.


• Experience cleaning and standardising data


• Experience matching, deduplicating or reconciling records across systems, whether rule-based or probabilistic.


• Understanding of how to measure data quality and match accuracy, including false positives and missed matches.


• Good working knowledge of UK GDPR and handling personal data securely.


• Use of version control (Git) and writing readable, documented, tested code.


• Ability to explain technical decisions and trade-offs clearly to non-technical colleagues.