Data Engineer: Azure Pipelines & Metadata-Driven Data

Perch Group; Lancashire; Hybrid; GBP 40,000 - 45,000; Full time

Benefits offered by this job

Hybrid work

Up to 20% annual bonus

Career development / formal training

Job summary

Perch Group is recruiting a Data Engineer to support the development of our modern data platform and drive metadata driven engineering. This role is ideal for someone with strong fundamentals in data engineering who is ready to grow into more advanced automation focused work.

You will build and maintain data pipelines using Azure Data Factory, Databricks, Python and T-SQL, apply parameterisation, and contribute to framework evolution.

Qualifications

• 3+ years T-SQL including joins, transformations and stored procedures.
• 3+ years Azure Data Factory, including pipeline development and templates.
• Experience with data warehouses and structured modelling concepts.
• Strong understanding of ETL/ELT practices and data centric engineering.
• Python experience.
• Exposure to metadata driven or config driven engineering practices.
• Understanding of data governance, data quality tooling, or lineage concepts.
• Experience in regulated environments or financial services.

Responsibilities

• Build, maintain, and enhance data pipelines using Azure Data Factory, Databricks, Python and T SQL.
• Apply parameterisation, configuration, and reusable components in ADF and Databricks where appropriate.
• Support the creation of metadata driven patterns by implementing components and contributing to framework evolution.
• Assist in developing and refining data models and data warehouse layers.
• Ensure pipelines are performant, reliable, and aligned with governance standards.
• Work closely with senior engineers, analysts, and architects to deliver robust, well designed solutions.
• Contribute to code reviews, documentation, and quality improvement initiatives.
• Participate in DevOps practices such as CI/CD, automated testing, and version control.
• Troubleshoot and resolve pipeline issues, ensuring stability and data accuracy.
• Monitor pipeline performance and help implement improvements.
• Support continuous optimisation and lifecycle management.

Skills

T-SQL

Azure Data Factory

Databricks

Python

ETL/ELT

Data governance

Regulated environments

Data modelling concepts

Tools

Azure Data Factory

Databricks

Job description

Perch Group is recruiting a Data Engineer to support the development of our modern data platform and drive metadata driven engineering. This role is ideal for someone with strong fundamentals in data engineering who is ready to grow into more advanced automation focused work.

You will build and maintain data pipelines using Azure Data Factory, Databricks, Python and T-SQL, apply parameterisation, and contribute to framework evolution.