Data Engineer

The Digital Recruitment Company; Greater London; Remote; GBP 60,000 - 90,000; Full time

Job summary

The Digital Recruitment Company is seeking an experienced Data Engineer to design, build, and support robust data pipelines and models. The role focuses on delivering reliable data platforms, governance, and analytics across the business, with emphasis on Microsoft Fabric, SFTP/API integrations, and strong SQL/Python capabilities.

Location options include Reading, Birmingham, London, and remote work (WFH). The role requires collaboration with Data Engineering and Governance teams to ensure data

Qualifications

• Strong hands-on data engineering experience with production pipelines.
• Microsoft Fabric experience including Lakehouse, Warehouse and OneLake.
• Advanced SQL skills for complex queries, transformations and troubleshooting.
• Strong Python and PySpark for production data workloads.
• Experience with ETL/ELT, data modelling, data quality and orchestration.
• Experience integrating internal and external data sources including APIs and SFTP.
• Git/Azure DevOps for source control and controlled deployments.
• Experience in regulated environments is advantageous.
• Clear documentation and communication with engineers and stakeholders.

Responsibilities

• Design, build, test and support data pipelines and data models.
• Document pipelines, models, integrations and processes; support maintainability.
• Integrate data from internal/external sources including API and SFTP.
• Build and optimise data transformations with data quality controls.
• Investigate data incidents and ensure platform reliability.
• Collaborate with Data Eng Manager, Data Product Manager and Enterprise Architect.
• Implement governance, privacy, security and resilience requirements.
• Contribute to code reviews, source control and deployment practices.
• Support existing data services and adoption of Fabric platform.
• Work with colleagues to meet business requirements and acceptance criteria.
• Identify opportunities to improve engineering quality and performance.

Skills

Data pipelines

SQL

Python

PySpark

Microsoft Fabric

ETL/ELT

Data governance

API integrations

SFTP integrations

Git

Azure DevOps

Documentation

Stakeholder comms

Wealth management domain

Tools

Git

Azure DevOps

Microsoft Fabric

Job description

Location: Reading/Birmingham/London & WFH

The Organisation

Join a top-tier wealth and advisory firm known for its robust infrastructure, cutting-edge technology, and unwavering commitment to service excellence. This dynamic organisation supports your professional growth, fosters collaborative teamwork, and promotes a culture where success and job satisfaction go hand in hand.

The Role

Our client are looking for an experienced Data Engineer to design, build, test, document and support reliable data pipelines and data models that enable trusted data and analytics across the business.

The role will deliver assigned Data Platform work to agreed priorities, technical designs and governance standards, while supporting the resolution of data incidents and the ongoing reliability of the platform.

Key Responsibilities

• Design, build, test and support assigned data pipelines and data models in line with agreed technical designs, engineering standards and priorities.
• Document data pipelines, data models, integrations and operational processes to support maintainability, knowledge sharing and effective support. Integrate data from internal and external source systems into the Data Platform, including SFTP and API-based integrations where required.
• Build and optimise data transformation, validation and loading processes, with appropriate controls for data quality and reliability.
• Investigate and resolve data incidents, defects and pipeline failures, escalating wider platform or design issues where appropriate.
• Work with the Data Engineering Manager, Data Product Manager and Enterprise Architect to confirm technical feasibility and deliver work in line with agreed architecture and design authority.
• Implement approved Data Governance, privacy, security and resilience requirements within assigned engineering deliverables.
• Contribute to code reviews, source control and deployment practices that support reliable and controlled change.
• Support existing data services and contribute to the continued development and adoption of the Microsoft Fabric Data Platform.
• Work collaboratively with Data Engineering, Data Governance, Technology and Change colleagues to deliver data products that meet agreed business requirements and acceptance criteria.
• Identify opportunities to improve engineering quality, automation and performance.

Experience & Skills Required

• Strong hands-on data engineering experience, building production data pipelines and data models.
• Strong hands-on Microsoft Fabric experience, including Lakehouse, Warehouse and OneLake, pipelines and notebooks.
• Advanced SQL skills, including complex querying, data transformation, optimisation and troubleshooting.
• Strong Python and PySpark skills, with experience developing and supporting production data transformation and processing workloads.
• Strong understanding of modern data engineering patterns, including ETL/ELT, medallion architecture, data modelling, data quality and orchestration.
• Experience integrating internal and external data sources, including APIs and SFTP.
• Experience with source control and controlled deployment practices, ideally using Git and Azure DevOps.
• Experience supporting production data services and diagnosing data, pipeline and integration incidents.
• Good understanding of data architecture and governance principles, with the ability to work within defined standards rather than own those standards.
• Clear documentation and communication skills, with the ability to work effectively with engineers, technical specialists and business stakeholders.
• Curious, motivated and pragmatic, with strong problem-solving skills and a clear sense of ownership for delivered work.
• Experience in wealth management, financial services or another highly regulated environment is advantageous.