Lead Data Engineer — Databricks & Platform Leader

Xcede; Greater London; Hybrid; GBP 110,000 - 150,000; Full time

Job summary

Xcede is partnering with a leading UK business to hire a Lead Data Engineer in London. You will lead a team of 4 data engineers, setting technical direction on scalable data products and pipelines, with Databricks on AWS at the core.

The role blends hands-on coding with people leadership and close collaboration with data science, analytics and product teams, while shaping a modern cloud data platform and driving data quality.

Qualifications

• Strong commercial Data Engineering experience.
• Previous experience leading, mentoring or managing Data Engineers.
• Strong Python and SQL skills.
• Experience with Databricks on AWS.
• Strong Spark and Airflow experience.
• Experience building scalable data pipelines and integrations.
• Good understanding of CI/CD and Git.
• Experience with Terraform or similar IaC tools.
• Strong understanding of data modelling and data quality.
• A genuine interest in developing people as well as technology.

Responsibilities

• Manage, mentor and develop a team of 4 Data Engineers.
• Provide technical leadership across data engineering projects.
• Design and deliver scalable data products and pipelines.
• Work closely with Data Scientists, Analysts, Architects and Product teams.
• Drive engineering standards around coding, testing and data quality.
• Oversee batch and real-time data integrations.
• Help shape the evolution of a modern cloud data platform.
• Improve engineering processes and ways of working across the wider data community.
• Provide hands-on technical guidance where required.
• Support wider data, ML and AI initiatives.

Skills

Leadership

Python

SQL

Databricks

Spark

Airflow

CI/CD

Git

Terraform

Data modelling

Data quality

Tools

Databricks on AWS

Airflow

Terraform

Job description

Xcede is partnering with a leading UK business to hire a Lead Data Engineer in London. You will lead a team of 4 data engineers, setting technical direction on scalable data products and pipelines, with Databricks on AWS at the core.

The role blends hands-on coding with people leadership and close collaboration with data science, analytics and product teams, while shaping a modern cloud data platform and driving data quality.