Lead Data Engineer

KDR Talent Solutions Ltd; Greater London; Hybrid; GBP 100,000 - 150,000; Full time

Benefits offered by this job

Equity options

Job summary

KDR Talent Solutions Ltd is seeking a Lead Data Engineer for a London-based AI startup operating in a hybrid model. You will own the data engineering direction and design the architecture to support AI-driven products. The role offers significant autonomy and opportunity to shape the platform from the ground up.

You will work closely with Backend, AI and Product Engineers, building scalable data systems while establishing reliability and performance practices across the data stack.

Qualifications

• Extensive experience building data systems for scalable AI-driven products.
• Strong problem-solving and leadership in data engineering.
• Proven ability to own architectural decisions and delivery.

Responsibilities

• Own Data Engineering and set technical direction.
• Design and build scalable data platforms for ingestion, processing and storage.
• Collaborate with Backend, AI and Product Engineers.
• Institute reliability, testing, observability and performance practices.
• Shape future data function and technology choices as the company grows.

Skills

5+ years in Data Engineering

Python

PostgreSQL

NoSQL

Data architectures

Distributed processing

Spark

Kafka

Airflow

Vector databases / AI data infra

Cloud platforms (GCP)

Job description

Lead Data Engineer | AI Startup | London (Hybrid) | £100,000 - £150,000 + Equity

KDR are working with a well-funded, London-based AI startup looking for a Lead Data Engineer to take ownership of their data engineering capability.

This is a particularly interesting opportunity for someone who enjoys building things from the ground up, working with highly technical people and having a genuine influence over the direction of a product.

The business is still relatively early in its journey, but has an experienced founding team, strong financial backing and ambitious plans for growth. They've invested heavily in engineering from the outset and are building sophisticated AI-driven software to tackle complex, real-world problems.

As Lead Data Engineer, you'll be the senior technical voice for Data Engineering, working closely with engineering leadership to design the architecture, build the underlying systems and establish how data engineering operates as the business scales.

This is a hands-on role with significant autonomy, suited to someone who wants to take real ownership rather than inherit an established platform and simply maintain it.

What you'll do

Own Data Engineering:

Take responsibility for the technical direction of Data Engineering, making key architectural decisions and establishing the foundations for future growth.

Build & Scale the Platform:

Design and develop scalable data systems, working across ingestion, processing, storage and the infrastructure supporting AI-driven products.

Work on Complex Engineering Problems:

Get involved in challenging areas of database design, data modelling, performance optimisation and distributed systems.

Partner with Engineering:

Work closely with Backend, AI and Product Engineers to ensure data architecture supports the wider product and its evolving requirements.

Introduce strong practices around reliability, testing, observability and performance, helping establish a mature engineering environment as the business grows.

Shape the Future Team:

Play an important role in defining the future direction of the data function, including the technologies, engineering standards and people needed as it expands.

What you'll bring

You'll have a strong background in Data Engineering, ideally with experience building complex data systems within a startup, scale-up or similarly demanding product engineering environment.

They're particularly interested in engineers with strong academic foundations, excellent technical problem-solving ability and a track record of progressing quickly and taking on greater responsibility.

• 5+ years of experience, predominantly within Data Engineering
• Strong Python skills and experience building production-grade data systems
• Deep understanding of relational databases, ideally PostgreSQL, alongside NoSQL technologies
• Experience designing data architectures, data models and scalable pipelines
• Knowledge of distributed processing and orchestration tools such as Spark, Kafka or Airflow
• Experience with vector databases, search technologies or data infrastructure supporting AI applications
• Strong understanding of database performance, optimisation and system reliability
• Experience making architectural decisions and owning technical delivery
• Exposure to cloud platforms such as GCP, AWS or Azure (They are on GCP)

Experience with AI/ML platforms, real-time data processing, vector search or complex backend engineering would be particularly interesting.

Beyond the technical skills, they're looking for someone ambitious, intellectually curious and comfortable taking ownership in an environment where priorities can change quickly.

Previous startup experience is particularly valuable. You'll need to be comfortable making decisions, challenging ideas and building solutions without always having an established process to follow.

Why this one is interesting

You'll be joining a business where engineering is central to the product, working alongside technically strong colleagues and founders with significant experience building and scaling businesses.

There's a real opportunity to influence the architecture from an early stage, establish the data engineering function and potentially grow into a broader leadership position as the business expands.

It's an ambitious environment with high expectations, but also the autonomy, backing and technical challenges to make it an exciting next step for the right person.

Package

• Equity options