Data Engineering Lead

DAC Beachcroft; United States; Remote; Confidential; Full time

Benefits offered by this job

Private medical insurance

Income protection insurance

Discounted gym membership

Job summary

DAC Beachcroft is seeking an experienced Data Engineering Lead to shape and deliver an enterprise data platform. You will lead design and delivery of scalable, secure, AI-ready data pipelines and products in a regulated environment.

You will work with Data Architecture, Analytics and business teams to operationalise advanced analytics, ensure data trust, and drive data-driven decisions across the organisation.

Qualifications

• Senior data engineering or lead role proven.
• Proficiency with Microsoft Azure data technologies and cloud data platforms.
• Strong Python and modern data orchestration experience.
• Experience building reliable production pipelines.
• Experience integrating data from multiple business systems.
• Knowledge of data security and access controls in cloud environments.
• Comfort working in regulated data environments with sensitive data.
• Excellent stakeholder management and communication skills.

Responsibilities

• Design, develop and maintain enterprise ETL/ELT pipelines integrating data from case management, finance, CRM and third-party systems.
• Lead the development and continuous improvement of our Azure-based data platform.
• Ensure pipelines are scalable, reliable, cost-effective and resilient.
• Build datasets supporting reporting, analytics, dashboards and data-enabled products.
• Establish and champion CI/CD, automated testing, version control, monitoring and alerting.
• Collaborate with Data Architects on target-state data models and modern data engineering principles.
• Partner with data scientists to operationalise analytics and predictive models.
• Manage integrations with insurer portals and external data providers with secure data flows.
• Support the organisation's ambition to be data-driven and AI-enabled.

Skills

Azure Data Factory

Azure Synapse

Azure Data Lake

Advanced SQL

Python

Data pipelines

CI/CD

Security & Compliance

Stakeholder management

Regulated data

Tools

Dynamics CRM

VisualFiles

Elite 3E

Job description

We are looking for an experienced Data Engineering Lead to play a pivotal role in shaping and delivering our enterprise data platform. This is an exciting opportunity to lead the development of scalable, secure, and AI-ready modern data platform that enable advanced analytics, reporting, automated and AI-enabled processes, and the creation of data powered digital products. Working closely with Data Architecture, Analytics, and wider business teams, you will be responsible for designing and maintaining robust data pipelines, integrating multiple business systems, and ensuring data remains trusted, accessible, consistent, and fit for purpose. This role is ideal for a technically strong data engineering professional who enjoys combining hands‑on delivery with leadership and continuous improvement.

What you will do?

• Design, develop and maintain enterprise ETL/ELT pipelines, integrating data from case management, finance, CRM and third‑party systems into a central data platform.
• Lead the development and continuous improvement of our cloud‑based data platform within Azure.
• Ensure data pipelines are scalable, reliable, cost‑effective and resilient.
• Build and maintain datasets that support business reporting, analytics, dashboards and data‑enabled products.
• Establish and champion engineering best practices, including CI/CD, automated testing, version control, monitoring and alerting on pipeline failures.
• Work closely with Data Architects to implement target‑state data models and modern data engineering principles.
• Partner with data scientists and analysts to operationalise advanced analytics and predictive models.
• Manage integrations with insurer portals and external data providers, ensuring high‑quality, timely and secure data flows.
• Support the organisation's ambition to become increasingly data‑driven and AI‑enabled.

Essential Experience & Skills

• Proven experience in a senior Data Engineering or Lead Data Engineer role.
• Strong hands‑on experience with Microsoft Azure data technologies, including: Azure Data Factory Azure Synapse Analytics Microsoft Fabric Azure Data Lake Advanced SQL development skills.
• Proficiency in Python and modern data pipeline orchestration tools, including their monitoring and automation.
• Track record of building reliable, monitored production‑grade data pipelines.
• Experience integrating data from multiple business‑critical applications and legacy systems.
• Good understanding of data security, data protection and access control principles within cloud‑based data platforms.
• Comfortable working in regulated data environments with sensitive personal and litigation data.
• Excellent stakeholder management and communication skills.

Desirable Experience

• Exposure to predictive analytics, machine learning operationalisation or AI platforms.
• Experience working with Dynamics CRM, case management platforms such as VisualFiles, and finance systems such as Elite 3E.
• Understanding of data modelling and data architecture best practices.
• Experience leading or mentoring data engineering teams.
• Experience within insurance, legal services, claims management or financial services is desirable but not mandatory.

Why Join Us?

This is an exciting opportunity to shape the future of data engineering within a growing, data‑driven organisation. You'll have the opportunity to:

• Be part of an exciting journey reshaping how data and AI are used in claims management.
• Play a key role in our major transformation initiatives and the creation of our modern cloud data platform.
• Work with cutting‑edge Azure and AI technologies.
• Influence strategic data decisions across the business.
• Deliver solutions that directly improve operational performance and customer outcomes.
• Join a collaborative and forward‑thinking Data & Analytics function.

Success Measures

• Reliability and performance of data pipelines and platform services.
• Delivery of high‑quality, trusted data products.
• Reduction in data‑related incidents and support issues.
• Improved data accessibility and adoption across the business.
• Successful implementation of engineering best practices and automation.
• Enablement of analytics, reporting and AI initiatives through scalable data pipelines and architecture.

What's in it for you?

• High levels of flexibility and a great work life balance – well‑rounded remuneration package (which includes private medical insurance, income protection insurance and discounted gym membership, amongst many other benefits).
• Opportunities for growth and progression including professional funding.
• In person and remote social events.
• Opportunity to get involved in a range of Environmental, Social and Governance (ESG) activities.
• We are dedicated to building a diverse, inclusive and authentic workplace, which aligns closely to our cultural principles (Determined, Clear, Creative and Supportive).
• We encourage you to apply even if your past experience does not align perfectly with every qualification in the job description – you may be the right candidate for this or other roles!

Department: CSG - Operations Hub (Data / Intelligence / Support) Employment Type: Permanent Workplace type: Fully remote