Data Engineering Lead, Azure Platform & AI Pipelines

DAC Beachcroft LLP; City Of London; On-site; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Private medical insurance

Income protection insurance

Discounted gym membership

Professional funding

Job summary

DAC Beachcroft LLP in London seeks a Data Engineering Lead to shape and deliver an enterprise data platform. You will design scalable, secure, AI-ready pipelines and collaborate with analytics, architecture and business teams.

Lead cloud migration on Azure, enforce CI/CD, data security, and governance while mentoring engineers and delivering data-powered products that drive decision-making. This role combines hands-on delivery with leadership, enabling advanced analytics and AI initiatives

Qualifications

• Proven experience in a senior Data Engineering or Lead Data Engineer role.
• Strong hands-on experience with Microsoft Azure data technologies (Azure Data Factory, Synapse Analytics, Microsoft Fabric, Data Lake).
• Advanced SQL development skills.
• Proficiency in Python and modern data pipeline orchestration tools.
• Track record of building reliable, production-grade data pipelines.
• Experience integrating data from multiple business-critical applications and legacy systems.
• Good understanding of data security, data protection and access control within cloud platforms.
• Experience in regulated data environments with sensitive data.
• Excellent stakeholder management and communication skills.

Responsibilities

• Design, develop and maintain enterprise ETL/ELT pipelines integrating data from multiple systems into a central data platform.
• Lead the development and continuous improvement of our cloud-based data platform within Azure.
• Ensure pipelines are scalable, reliable, cost-effective and resilient.
• Build and maintain datasets for reporting, analytics, dashboards and data-enabled products.
• Establish and champion engineering best practices including CI/CD, automated testing and monitoring.
• Collaborate with Data Architects to implement target-state data models and modern data principles.
• Partner with data scientists to operationalise advanced analytics and predictive models.
• Manage integrations with insurer portals and external providers ensuring secure data flows.
• Support the organisation's ambition to be data-driven and AI-enabled.

Skills

Senior Data Engineer

Azure Data Factory

Azure Synapse Analytics

Microsoft Fabric

Azure Data Lake

Advanced SQL

Python

Data pipelines

Stakeholder management

Job description

DAC Beachcroft LLP in London seeks a Data Engineering Lead to shape and deliver an enterprise data platform. You will design scalable, secure, AI-ready pipelines and collaborate with analytics, architecture and business teams.

Lead cloud migration on Azure, enforce CI/CD, data security, and governance while mentoring engineers and delivering data-powered products that drive decision-making. This role combines hands-on delivery with leadership, enabling advanced analytics and AI initiatives