Lead Data Engineer

Arrows; Greater London; On-site; GBP 120,000 - 180,000; Full time

Job summary

Arrows is hiring a Lead Data Engineer in London to own the data architecture from day one, shaping scalable pipelines for AI products in a fast-growing startup.

You will work with AI engineers, backend engineers and product teams to translate business needs into real-time, AI-ready data infrastructure while mentoring a small team as the company scales towards a category-defining platform.

Qualifications

• 7+ years of experience in data engineering or backend engineering.
• Experience designing and building data pipelines and distributed data systems.
• Experience with relational databases (PostgreSQL preferred) and NoSQL databases.
• Experience with vector databases used in modern AI systems.
• Strong programming experience in Python.

Responsibilities

• Architecting and building scalable data pipelines and infrastructure to support AI and product systems
• Designing data ingestion, transformation and storage architectures for operational and AI workloads
• Developing and managing batch and real-time data pipelines
• Building and optimising systems for vector search, retrieval and ML data pipelines
• Ensuring data reliability, security and governance across the platform
• Implementing monitoring, observability and data quality frameworks
• Contributing to technical architecture decisions and long-term data strategy
• Helping build and mentor the future data engineering team as the company scales

Skills

Data architecture

Data pipelines design

Distributed systems

Startup environment

Tools

Python

PostgreSQL

MongoDB

Qdrant

Milvus

pgvector

Apache Spark

Apache Airflow

Kafka

Elasticsearch/OpenSearch

Pandas

Polars

Job description

I'm currently partnered with an early-stage, well-funded AI business that is building the operating system for a large, traditionally under-digitised industry, and looking to hire their first Lead Data Engineer into a small, high-impact team.

This is a broad and genuinely technical role sitting at the heart of the data function, responsible for architecting and scaling the systems powering their AI products. Already live with pilot customers managing 14,000+ units, this is a rare opportunity to join early and own the data architecture from day one, working closely with AI engineers, backend engineers and product teams to translate business needs into real-time, AI-ready data infrastructure.

The environment suits someone who enjoys building from scratch, takes pride in architectural rigour, and wants to be the defining technical voice behind a company's data foundations as it scales toward a category-defining AI platform.

What you'll be doing:

• Architecting and building scalable data pipelines and infrastructure to support AI and product systems
• Designing data ingestion, transformation and storage architectures for operational and AI workloads
• Developing and managing batch and real-time data pipelines
• Building and optimising systems for vector search, retrieval and ML data pipelines
• Ensuring data reliability, security and governance across the platform
• Implementing monitoring, observability and data quality frameworks
• Contributing to technical architecture decisions and long-term data strategy
• Helping build and mentor the future data engineering team as the company scales

Python, PostgreSQL, MongoDB, vector databases (Qdrant, Milvus or pgvector), Apache Spark, Apache Airflow, Kafka, Elasticsearch/OpenSearch, Pandas, Polars.

What they're looking for:

• 7+ years of experience in data engineering or backend engineering
• Strong experience designing and building data pipelines and distributed data systems
• Experience with relational databases (PostgreSQL preferred) and NoSQL databases
• Experience with vector databases used in modern AI systems
• Strong programming experience in Python
• Comfortable working in a fast-moving, high-ownership startup environment