Senior Data Engineer - AI Platform & Data Architecture (Remote)

Smart Working Solutions; United Kingdom; Remote; GBP 95,000 - 150,000; Full time

Job summary

Smart Working Solutions is seeking a Lead Data Engineer to architect and scale the data infrastructure powering a cutting-edge AI assistant for property management. You will own real-time data pipelines, vector databases and ML data workflows, shaping architecture and data team culture as the first senior data hire.

You will collaborate with AI, backend, and product teams to ensure scalable, reliable data platforms and guide long-term data strategy across global teams.

Qualifications

• 7+ years of professional experience in data engineering.
• Experience designing and building data pipelines and distributed data systems.
• Experience with relational and NoSQL databases.
• Experience with vector databases used in AI.
• Strong programming in Python.
• Ability to justify architectural decisions.
• Experience building scalable backend systems.
• Experience with data models and storage architectures.
• Strong understanding of data processing performance and optimisation.
• Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch; PostgreSQL, MongoDB, vector databases.

Responsibilities

• Architect and build scalable data pipelines and infrastructure to support AI and product systems.
• Design and maintain data ingestion, transformation and storage architectures for operational and AI workloads.
• Develop and manage batch and real-time data pipelines.
• Build and optimise systems for vector search, retrieval and ML data pipelines.
• Ensure data reliability, security and governance across the platform.
• Collaborate with AI and backend engineering teams to support training, inference and product features.
• Implement monitoring, observability and data quality frameworks.
• Optimise performance of large-scale datasets and query systems.
• Contribute to technical architecture decisions and long-term data strategy.
• Act as the founding data hire, defining culture, standards and hiring bar as the data function scales.
• Partner with founders and product leadership to translate data capabilities into product decisions.

Skills

Python

PostgreSQL

NoSQL

Vector databases

Data pipelines

Backend systems

Architectural decisions

Spark

Airflow

Kafka

Elasticsearch/OpenSearch

pgvector / Milvus / Qdrant

Pandas/Polars

Tools

PostgreSQL

Job description

Smart Working Solutions is seeking a Lead Data Engineer to architect and scale the data infrastructure powering a cutting-edge AI assistant for property management. You will own real-time data pipelines, vector databases and ML data workflows, shaping architecture and data team culture as the first senior data hire.

You will collaborate with AI, backend, and product teams to ensure scalable, reliable data platforms and guide long-term data strategy across global teams.