Lead Data Engineer

Xcede; England; Hybrid; Full time

Benefits offered by this job

Flexible hybrid working approach

Opportunity for leadership responsibilities with technical delivery

Influence over next-generation data and AI capabilities

Job summary

A major financial services organisation is recruiting for a Lead Data Engineer in South England. This role combines technical leadership with hands-on engineering, overseeing the design and delivery of scalable data solutions. Candidates should have experience leading data engineering teams and a strong background in cloud data platforms. This opportunity offers a flexible hybrid work environment and significant influence over data and AI capabilities in a renowned firm.

Qualifications

• Proven background leading data engineering teams or projects in a technology-driven business.
• Expert knowledge of modern cloud data platforms like Databricks, Snowflake, ideally AWS.
• Advanced Python programming skills and fluency with the wider Python data toolkit.

Responsibilities

• Directing major data engineering initiatives and overseeing a team of engineers.
• Designing and enhancing data warehouses, pipelines, and integration frameworks at scale.
• Partnering with analytics, AI, product, and business teams.

Skills

Leading data engineering teams or projects

Expert knowledge of modern cloud data platforms (Databricks, Snowflake, AWS)

Advanced Python programming skills

Strong capability with SQL, Spark, Airflow, Terraform

Understanding of CICD practices

Experience building data integration pipelines

Data modelling, ingestion, and optimisation

Exposure to Generative AI

Job description

Overview

Associate Director – Data & AI Recruitment

Location: South England x2 days a month in office. Package: Competitive salary + strong benefits.

We’re representing a major financial services organisation that is investing heavily in next-generation data and digital platforms. They are building a modern, cloud-native data ecosystem to power AI, analytics, and customer-facing innovation and now seek a Lead Data Engineer to guide this transformation. The company has an incredible Data Leadership team with good tenure, and alongside impactful Data Science projects, they have multiple AI / LLM based projects in production due to their mature Data offering. This is a rare opportunity to join the team at an impactful, senior level due to the typically very long tenure.

The role

This position blends technical leadership with hands-on engineering. You’ll oversee the design and delivery of scalable data solutions, lead and mentor a small engineering group, and set the standards for how data is built, managed, and consumed across the organisation.

It’s a chance to make architectural decisions, work directly with product and AI specialists, and drive forward best practices in a rapidly modernising environment.

Key responsibilities

• Directing major data engineering initiatives and overseeing a team of engineers
• Designing and enhancing data warehouses, pipelines, and integration frameworks at scale
• Partnering with analytics, AI, product, and business teams to create end-to-end data solutions
• Defining coding guidelines, frameworks, and operating models across the platform
• Leading the migration of legacy environments into a flexible, cloud-first stack
• Contributing to the build and delivery of advanced data products and GenAI-enabled applications
• Raising the bar on data quality, governance, and compliance processes

Skills & experience required

• Proven background leading data engineering teams or projects in a technology-driven business
• Expert knowledge of modern cloud data platforms (Databricks, Snowflake, ideally AWS)
• Advanced Python programming skills and fluency with the wider Python data toolkit
• Strong capability with SQL, Spark, Airflow, Terraform, and workflow orchestration tools
• Solid understanding of CICD practices using Git, Jenkins, or equivalent tooling
• Hands-on experience building both batch and streaming data integration pipelines
• Depth in data modelling, ingestion, and optimisation for large-scale environments
• Any exposure to production use of Generative AI is highly advantageous

What’s on offer

• A hybrid working approach that values flexibility
• The opportunity to combine leadership responsibilities with technical delivery
• Influence over the design of next-generation data and AI capabilities within a household-name financial services brand