Azure Data Engineer: Fabric Pipelines & Data Quality

Arrow Global Group; Manchester; On-site; GBP 60,000 - 90,000; Full time

Job summary

Arrow Global Group in Manchester seeks an experienced Data Engineer to design and implement robust data pipelines using Microsoft Fabric, PySpark, and SQL across multiple data sources in a financial services setting.

The role focuses on building scalable data ingestion, transformation, and validation workflows, while collaborating with Data Stewards, Analysts, and the Data Architect to deliver auditable, trusted data and Power BI models.

Qualifications

• Experience building data pipelines as a Data Engineer or in a similar role.
• Strong Python/PySpark and SQL skills.
• Hands-on experience with Microsoft Fabric, or with Databricks / Azure Synapse / Azure Data Factory and a willingness to move to Fabric.
• Comfortable with Git, pull requests and CI/CD. Terraform or other infrastructure-as-code experience is a plus.
• Understanding of data modelling (dimensional / medallion) and data quality practices.

Responsibilities

• Build and maintain ingestion and transformation pipelines on Microsoft Fabric (Data Pipelines, PySpark notebooks, Dataflows Gen2) across the Landing, Curate and Enrich layers of the medallion architecture.
• Onboard new source systems by setting up the team’s reusable ingestion patterns (Fabric mirroring from Snowflake and SQL Server, change data capture, file and API loads). Do this through configuration rather than one-off code.
• Write and tune PySpark and T-SQL for Lakehouse and Warehouse workloads, including fact and dimension models in the enterprise data warehouse.
• Implement data quality rules and validation pipelines, and help investigate and fix data issues found by DQ checks and reconciliations.
• Work through the team’s engineering workflow: Git-based development in Azure DevOps, pull requests and peer review, and CI/CD promotion from Dev to UAT to Production.
• Contribute to the infrastructure-as-code (Terraform) and the orchestration config database (SQL projects / DACPAC) as needed.
• Work with Data Stewards, Data Analysts and the Data Architect to understand source data and turn mapping specs and business rules into working transformations.
• Keep documentation current, including source-to-target mappings, transformation rules and runbooks, so the work stays auditable and supportable.
• Support the platform day to day: monitor pipeline runs, triage failures and escalates where needed.
• Help prepare trusted datasets and Power BI semantic models for portfolio analytics, asset strategy, finance and risk.

Skills

Python

PySpark

SQL

Microsoft Fabric

Databricks

Azure Data Factory

Azure Synapse

Education

Degree in Computer Science, Information Systems, Data Science

Tools

Microsoft Fabric

Databricks

Azure Data Factory

Azure Synapse

Job description

Arrow Global Group in Manchester seeks an experienced Data Engineer to design and implement robust data pipelines using Microsoft Fabric, PySpark, and SQL across multiple data sources in a financial services setting.

The role focuses on building scalable data ingestion, transformation, and validation workflows, while collaborating with Data Stewards, Analysts, and the Data Architect to deliver auditable, trusted data and Power BI models.