Lead Data Engineer

SF Recruitment; West Midlands; On-site; GBP 60,000 - 85,000; Full time

Job summary

A leading recruitment firm is seeking a Data Architect for a market-leading organization in West Midlands. This role involves designing and implementing an Azure-based data platform, focusing on scalable data architecture and governance. The ideal candidate will have over 3 years of experience in data architecture, especially with cloud platforms, and a strong grasp of data governance and modelling techniques.

Qualifications

• 3+ years in a Data Architect or senior data engineering role.
• Proven record of designing cloud-based data platforms, ideally in Azure.
• Experience leading data transformation initiatives.

Responsibilities

• Design and implement an enterprise data lake on Azure.
• Build and maintain scalable ETL/ELT pipelines in Azure Data Factory.
• Lead the build, testing, and deployment of the Azure data platform.

Skills

Azure Data Lake Gen2

Azure Data Factory

Azure Active Directory

Data modelling skills

SQL for complex transformations

Understanding of data governance frameworks

Tools

Power BI

ODBC connectors

REST APIs

Job description

About the Role

We re working with a market-leading organisation that s undergoing a major transformation, moving from manual, Excel-based reporting to a fully automated, intelligence-driven data ecosystem.

As Data Architect, you ll be responsible for designing and implementing the Azure-based data platform that becomes the single source of truth across the business. This is a hands-on, strategic role where you ll build scalable, governed data architecture and shape how data is used across Finance, Operations, and Commercial functions.

What You ll Be Doing

Data Architecture & Platform Design

• Design and implement an enterprise data lake on Azure Data Lake Gen2, using Bronze/Silver/Gold architecture.

• Build and maintain scalable ETL/ELT pipelines in Azure Data Factory to integrate data from core systems (AS400, Tagetik, CRM, Esker, Slimstock).

• Develop the overall data model, data dictionaries, and lineage documentation.

• Deliver a stable batch-first integration strategy with AS400 during its .NET migration, with a roadmap toward API integration.

Data Governance & Quality

• Implement the technical foundation for data governance quality checks, metadata management, and master data validation.

• Embed business rules and validation logic directly within data pipelines.

• Define and manage data security and access controls (Azure and Power BI row-level security).

Implementation & Optimisation

• Lead the hands-on build, testing, and deployment of the Azure data platform.

• Monitor platform performance and optimise pipelines for cost, scalability, and speed.

• Define and document technical standards and best practices.

• Oversee the migration from legacy tools (Domo, Vecta) to the new Power BI ecosystem.

What You ll Bring

Technical Skills

• Strong hands-on experience with Azure Data Lake Gen2, Azure Data Factory, and Azure Active Directory.

• Advanced skills in data modelling (conceptual, logical, physical) and SQL for complex transformations.

• Proven ability to design and build high-performance ETL/ELT pipelines.

• Understanding of data governance, security, and access control frameworks.

• Knowledge of batch and real-time data integration and experience with ODBC connectors or REST APIs.

• Familiarity with Databricks and/or Microsoft Fabric is a bonus.

Experience

• 3+ years in a Data Architect or senior data engineering role.

• Proven record of designing and delivering cloud-based data platforms, ideally in Azure.

• Background working with complex ERP or transactional systems.

• Experience supporting or leading data transformation initiatives within a business setting.