Data Engineering Lead: AI-Ready Cloud Platform

DAC Beachcroft LLP; Greater London; Hybrid; GBP 90,000 - 140,000; Full time

Benefits offered by this job

Private medical insurance

Income protection insurance

Gym membership

Job summary

DAC Beachcroft LLP is seeking an experienced Data Engineering Lead in London to shape and deliver an enterprise data platform. You will design and maintain robust ETL/ELT pipelines across case management, finance and CRM systems, driving a cloud-first data strategy with AI readiness.

Engage with Data Architects, analysts and data scientists to operationalise advanced analytics, while leading a team and embedding CI/CD, monitoring and security best practices.

Qualifications

• Proven experience in a senior Data Engineering or Lead Data Engineer role.
• Strong hands-on experience with Microsoft Azure data technologies including ADF, Synapse, Fabric and Data Lake.
• Advanced SQL development skills.
• Proficiency in Python and modern data pipeline orchestration tools with monitoring/automation.
• Track record of building reliable, production-grade data pipelines.
• Experience integrating data from multiple business-critical applications and legacy systems.
• Good understanding of data security, data protection and access control in cloud platforms.
• Comfortable in regulated data environments with sensitive data.
• Excellent stakeholder management and communication skills.

Responsibilities

• Design, develop and maintain enterprise ETL/ELT pipelines across multiple systems.
• Lead the development of a cloud-based data platform on Azure.
• Ensure data pipelines are scalable, reliable, and cost-effective.
• Build datasets for reporting, analytics and AI-enabled products.
• Establish engineering best practices including CI/CD, testing, version control, monitoring.
• Collaborate with Data Architects on target-state models and modern concepts.
• Partner with data scientists to operationalise analytics and models.
• Manage integrations with insurer portals and external providers securely.
• Support the shift to a more data-driven and AI-enabled organisation.

Skills

Senior Data Engineer

Azure Data Factory

Azure Synapse Analytics

Microsoft Fabric

Azure Data Lake

Advanced SQL

Python

Data Orchestration

CI/CD

Production-grade pipelines

Data security & privacy

Stakeholder management

Regulated data environments

Tools

Azure Data Factory

Azure Synapse

Microsoft Fabric

Azure Data Lake

Job description

DAC Beachcroft LLP is seeking an experienced Data Engineering Lead in London to shape and deliver an enterprise data platform. You will design and maintain robust ETL/ELT pipelines across case management, finance and CRM systems, driving a cloud-first data strategy with AI readiness.

Engage with Data Architects, analysts and data scientists to operationalise advanced analytics, while leading a team and embedding CI/CD, monitoring and security best practices.