Lead Data Engineer

SF Recruitment; Minworth; On-site; GBP 60,000 - 80,000; Full time

Job summary

A leading recruitment agency is looking for a Data Architect to design and implement an Azure-based data platform that serves as the single source of truth across the business. The ideal candidate will have a strong background in building scalable data architectures, managing ETL/ELT pipelines, and ensuring data governance. This hands-on strategic role requires at least 3 years of experience in a similar position.

Qualifications

• 3+ years in a Data Architect or senior data engineering role.
• Proven record of designing and delivering cloud-based data platforms, ideally in Azure.
• Experience supporting or leading data transformation initiatives within a business setting.

Responsibilities

• Design and implement an enterprise data lake on Azure Data Lake Gen2.
• Lead the hands-on build, testing, and deployment of the Azure data platform.
• Define and manage data security and access controls within Azure and Power BI.

Skills

Azure Data Lake Gen2

Azure Data Factory

SQL

Data Governance

ETL/ELT pipelines

Data modelling

Data Security

Tools

Power BI

Databricks

ODBC connectors

Job description

About the Role

We're working with a market-leading organisation that's undergoing a major transformation, moving from manual, Excel-based reporting to a fully automated, intelligence-driven data ecosystem.

As Data Architect, you'll be responsible for designing and implementing the Azure-based data platform that becomes the single source of truth across the business. This is a hands-on, strategic role where you'll build scalable, governed data architecture and shape how data is used across Finance, Operations, and Commercial functions.

What You'll Be Doing

• Data Architecture & Platform Design
• Design and implement an enterprise data lake on Azure Data Lake Gen2, using Bronze/Silver/Gold architecture.
• Build and maintain scalable ETL/ELT pipelines in Azure Data Factory to integrate data from core systems (AS400, Tagetik, CRM, Esker, Slimstock).
• Develop the overall data model, data dictionaries, and lineage documentation.
• Deliver a stable “batch-first” integration strategy with AS400 during its .NET migration, with a roadmap toward API integration.

• Data Governance & Quality
• Implement the technical foundation for data governance - quality checks, metadata management, and master data validation.
• Embed business rules and validation logic directly within data pipelines.
• Define and manage data security and access controls (Azure and Power BI row-level security).

• Implementation & Optimisation
• Lead the hands-on build, testing, and deployment of the Azure data platform.
• Monitor platform performance and optimise pipelines for cost, scalability, and speed.
• Define and document technical standards and best practices.
• Oversee the migration from legacy tools (Domo, Vecta) to the new Power BI ecosystem.

What You'll Bring

• Technical Skills Strong hands-on experience with Azure Data Lake Gen2, Azure Data Factory, and Azure Active Directory.
• Advanced skills in data modelling (conceptual, logical, physical) and SQL for complex transformations.
• Proven ability to design and build high-performance ETL/ELT pipelines.
• Understanding of data governance, security, and access control frameworks.
• Knowledge of batch and real-time data integration and experience with ODBC connectors or REST APIs.
• Familiarity with Databricks and/or Microsoft Fabric is a bonus.

Experience

• 3+ years in a Data Architect or senior data engineering role.
• Proven record of designing and delivering cloud-based data platforms, ideally in Azure.
• Background working with complex ERP or transactional systems.
• Experience supporting or leading data transformation initiatives within a business setting