Data Engineering Lead

DAC Beachcroft; Greater London; On-site; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Private medical insurance

Income protection

Flexible working

Professional funding

Remote social events

ESG activities

Job summary

DAC Beachcroft in London – Woolgate is seeking a Data Engineering Lead to shape and deliver an enterprise data platform. You will design and maintain scalable ETL/ELT pipelines, lead cloud adoption on Azure, and partner with data scientists to operationalise analytics and AI-enabled processes.

Excellent data security, governance and stakeholder management are essential. Join a collaborative team driving data-driven decisions across claims, finance and CRM integrations, while building trusted

Qualifications

• Senior Data Engineer or Lead Data Engineer with proven experience.
• Hands-on with Azure data technologies (ADF, Synapse, Data Lake).
• Strong SQL development and Python skills, plus data pipeline orchestration.

Responsibilities

• Design, develop and maintain enterprise ETL/ELT pipelines integrating data from multiple systems.
• Lead the cloud-based data platform development within Azure.
• Ensure pipelines are scalable, reliable, and cost-efficient.

Skills

Azure Data Factory

Azure Synapse Analytics

Microsoft Fabric

Azure Data Lake

SQL development

Python

Data pipelines

Data security

Stakeholder management

Tools

VisualFiles

Elite 3E

Dynamics CRM

Job description

Department: CSG - Operations Hub (Data / Intelligence / Support)

Location: London - Woolgate

Description

We are looking for an experienced Data Engineering Lead to play a pivotal role in shaping and delivering our enterprise data platform. This is an exciting opportunity to lead the development of scalable, secure, and AI-ready modern data platform that enable advanced analytics, reporting, automated and AI-enabled processes, and the creation of data powered digital products. Working closely with Data Architecture, Analytics, and wider business teams, you will be responsible for designing and maintaining robust data pipelines, integrating multiple business systems, and ensuring data remains trusted, accessible, consistent, and fit for purpose. This role is ideal for a technically strong data engineering professional who enjoys combining hands‑on delivery with leadership and continuous improvement.

What you will do?

• Design, develop and maintain enterprise ETL/ELT pipelines, integrating data from case management, finance, CRM and third‑party systems into a central data platform.
• Lead the development and continuous improvement of our cloud-based data platform within Azure.
• Ensure data pipelines are scalable, reliable, cost‑effective and resilient.
• Build and maintain datasets that support business reporting, analytics, dashboards and data‑enabled products.
• Establish and champion engineering best practices, including CI/CD, automated testing, version control, monitoring and alerting on pipeline failures.
• Work closely with Data Architects to implement target‑state data models and modern data engineering principles.
• Partner with data scientists and analysts to operationalise advanced analytics and predictive models.
• Manage integrations with insurer portals and external data providers, ensuring high‑quality, timely and secure data flows.
• Support the organisation's ambition to become increasingly data‑driven and AI‑enabled.

Who you are

Essential Experience & Skills

• Proven experience in a senior Data Engineering or Lead Data Engineer role.
• Strong hands‑on experience with Microsoft Azure data technologies, including:
  • Azure Data Factory
  • Azure Synapse Analytics
  • Microsoft Fabric
  • Azure Data Lake
• Advanced SQL development skills.
• Proficiency in Python and modern data pipeline orchestration tools, including their monitoring and automation.
• Track record of building reliable, monitored production‑grade data pipelines.
• Experience integrating data from multiple business‑critical applications and legacy systems.
• Good understanding of data security, data protection and access control principles within cloud‑based data platforms.
• Comfortable working in regulated data environments with sensitive personal and litigation data.
• Excellent stakeholder management and communication skills.

Desirable Experience

• Exposure to predictive analytics, machine learning operationalisation or AI platforms.
• Experience working with Dynamics CRM, case management platforms such as VisualFiles, and finance systems such as Elite 3E.
• Understanding of data modelling and data architecture best practices.
• Experience leading or mentoring data engineering teams.
• Experience within insurance, legal services, claims management or financial services is desirable but not mandatory.

Why Join Us?

This is an exciting opportunity to shape the future of data engineering within a growing, data‑driven organisation.

• Be part of an exciting journey reshaping how data and AI are used in claims management.
• Play a key role in our major transformation initiatives and the creation of our modern cloud data platform.
• Work with cutting‑edge Azure and AI technologies.
• Influence strategic data decisions across the business.
• Deliver solutions that directly improve operational performance and customer outcomes.
• Join a collaborative and forward‑thinking Data & Analytics function.

Success Measures

• Reliability and performance of data pipelines and platform services.
• Delivery of high‑quality, trusted data products.
• Reduction in data‑related incidents and support issues.
• Improved data accessibility and adoption across the business.
• Successful implementation of engineering best practices and automation.
• Enablement of analytics, reporting and AI initiatives through scalable data pipelines and architecture.

Ready to Apply?

If you are passionate about building modern data platforms and turning complex data challenges into business value, we'd love to hear from you. Join us and play a key role in shaping the future of data across the organisation.

What's in it for you?

• High levels of flexibility and a great work life balance - https://www.dacbeachcroft.com/en/Work-with-us/Whats-in-it-for-you
• A well‑rounded remuneration package (which includes private medical insurance, income protection insurance and discounted gym membership, amongst many other benefits)
• Opportunities for growth and progression including professional funding
• In person and remote social events
• Opportunity to get involved in a range of Environmental, Social and Governance (ESG) activities

We are dedicated to building a diverse, inclusive and authentic workplace, which aligns closely to our cultural principles (Determined, Clear, Creative and Supportive). If you are excited about this role and being part of our culture, but your past experience does not align perfectly with every qualification in the job description, we encourage you to apply anyway. You may be just the right candidate for this or other roles!

We are happy to talk flexible working with our Flex Forward scheme. We would encourage you to talk to us about our approach to flexible working during the hiring process if you would like to explore this further.