Lead Data Engineer

LightWork AI; Greater London; On-site; GBP 90,000 - 140,000; Full time

Job summary

LightWork AI, based in London, is hiring a Lead Data Engineer to design and scale the data backbone of the platform. You will architect data infrastructure powering AI products, from real-time pipelines to vector databases, and work with AI and backend teams to enable reliable, intelligent data processing.

You will lead as the first senior data hire, defining data architecture, tooling, and standards while shaping the future data team and culture at LightWork AI.

Qualifications

• 7+ years in dedicated data engineering roles.
• Experience designing and building data pipelines and distributed data systems.
• Experience with relational databases (PostgreSQL preferred).
• Experience with NoSQL databases and vector databases.

Responsibilities

• Architect and build scalable data pipelines and infrastructure to support LightWork AI’s AI and product systems.
• Design and maintain data ingestion, transformation, and storage architectures for operational and AI workloads.
• Develop and manage batch and real‑time data pipelines.
• Build and optimize systems for vector search, retrieval, and ML data pipelines.
• Ensure data reliability, security, and governance across the platform.
• Collaborate with AI and backend engineering teams to support training, inference, and product features.
• Implement monitoring, observability, and data quality frameworks.
• Optimize performance of large-scale datasets and query systems.
• Contribute to technical architecture decisions and long-term data strategy.
• Act as the founding data hire, defining culture, standards and hiring bar for the data function as it scales
• Partner directly with founders and product leadership to translate data capabilities into product decisions.

Skills

Data pipelines

Distributed data systems

Python

Architectural decisions

Data governance

Performance optimization

Problem solving

Tools

Apache Spark

Apache Airflow

Kafka

Elasticsearch/OpenSearch

Job description

Location: London | Full-time | 4 days in the office per week

About LightWork AI

Lead Data Engineer

Location: London | Full-time | 4 days in the office per week

About LightWork AI

LightWork AI is building the operating system for modern property management.

Our platform, Felicity, is an intelligent AI assistant that automates the operational workload that slows down property managers, letting agents and build‑to‑rent teams. Felicity manages communications, compliance, maintenance coordination and scheduling — enabling property businesses to operate with the efficiency of modern digital platforms.

The UK property management services industry is a £33B market undergoing a structural shift toward automation. Property teams currently spend up to 75% of their time on repetitive operational admin, and our AI is designed to automate up to 80–90% of these workflows.

We are already live with pilot customers, with overwhelmingly positive feedback and strong demand for the product as we scale toward revenue in 2026.

LightWork AI is backed by experienced founders, industry advisors, and investors, and we are building toward a clear goal: to create the defining AI platform for property operations and build a unicorn company in the process.

This is a rare opportunity to join early and own the data architecture of a fast‑scaling AI company building the category‑defining platform for property operations.

The Role

We are looking for a Lead Data Engineer to build and lead the data infrastructure powering LightWork AI’s platform.

This role will architect and scale the systems that power our AI products — from real‑time data pipelines and analytics infrastructure to vector databases and machine learning data workflows.

You will work closely with AI engineers, backend engineers, and product teams to ensure our platform can process large volumes of operational data reliably and intelligently.

As the first senior data hire, you will define our data architecture, tooling, and engineering standards, and play a key role in building the foundations of the future data team.

Key Responsibilities

• Architect and build scalable data pipelines and infrastructure to support LightWork’s AI and product systems.
• Design and maintain data ingestion, transformation, and storage architectures for operational and AI workloads.
• Develop and manage batch and real‑time data pipelines.
• Build and optimize systems for vector search, retrieval, and ML data pipelines.
• Ensure data reliability, security, and governance across the platform.
• Collaborate with AI and backend engineering teams to support training, inference, and product features.
• Implement monitoring, observability, and data quality frameworks.
• Optimize performance of large-scale datasets and query systems.
• Contribute to technical architecture decisions and long-term data strategy.
• Act as the founding data hire, defining culture, standards and hiring bar for the data function as it scales
• Partner directly with founders and product leadership to translate data capabilities into product decisions.

Requirements

Core Experience

• 7+ years with the majority in dedicated data engineering roles
• Strong experience designing and building data pipelines and distributed data systems.
• Experience working with relational databases (PostgreSQL preferred, but MySQL or similar is acceptable).
• Experience with NoSQL databases.
• Experience with vector databases used in modern AI systems.
• Strong programming experience in Python.
• Demonstrated ability to make and justify architectural decisions, not just implement them.

Data & Infrastructure

Frameworks / Infrastructure

• Apache Spark
• Apache Airflow
• Kafka
• Elasticsearch / OpenSearch

Databases

• PostgreSQL
• MongoDB
• Vector databases such as Qdrant, Milvus, or pgvector

Python Libraries

• Pandas
• Polars

Engineering Skills

• Experience building scalable backend systems.
• Experience designing data models and storage architectures.
• Strong understanding of data processing performance and optimization.

Nice to Have

• Experience working on AI or machine learning platforms.
• Familiarity with stream processing and event‑driven architectures.
• Experience with cloud infrastructure (GCP, AWS, or Azure).
• Experience working in high‑growth startups or early‑stage companies.

What We’re Looking For

• A high‑ownership engineer who enjoys building systems from scratch.
• Someone excited to design the data backbone of an AI company.
• Strong problem‑solving ability and architectural thinking.
• Comfortable working in a fast‑moving startup environment.
• Motivated by the opportunity to build technology that transforms an entire industry.

Why Join LightWork AI

• Build the data foundations of a category‑defining AI company.
• Work directly with the founders and core engineering team.
• Join at an early stage with massive ownership and influence.
• Competitive salary + equity — share in the upside of what we build.
• Help create the AI infrastructure powering the future of property operations.