AI Data Infrastructure Lead — Remote-First

Smart Working; Greater London; Remote; GBP 90,000 - 140,000; Full time

Job summary

Smart Working is seeking a Lead Data Engineer to build and scale the data infrastructure powering its AI assistant platform. You will architect real-time data pipelines, analytics, vector databases, and ML data workflows, working closely with AI and backend engineers to ensure reliable, scalable data processing across teams.

You will define data architecture, tooling, and standards, and play a founding role in shaping the data function as the company grows.

Qualifications

• 7+ years in dedicated data engineering roles.
• Designing and building data pipelines and distributed systems.
• Experience with PostgreSQL and NoSQL databases.
• Strong Python programming skills.
• Ability to justify architectural choices and drive data strategy.

Responsibilities

• Architect and scale data pipelines and infrastructure for AI-enabled platforms.
• Define data architecture, tooling, and engineering standards.
• Collaborate with AI and backend teams to support training and inference.
• Implement monitoring and data quality frameworks.
• Contribute to long-term data strategy and hiring.

Skills

Data engineering

Python programming

Data pipelines

Distributed systems

Architectural decision making

Tools

PostgreSQL

NoSQL databases

Spark

Airflow

Kafka

Elasticsearch/OpenSearch

Qdrant/Milvus/pgvector

Job description

Smart Working is seeking a Lead Data Engineer to build and scale the data infrastructure powering its AI assistant platform. You will architect real-time data pipelines, analytics, vector databases, and ML data workflows, working closely with AI and backend engineers to ensure reliable, scalable data processing across teams.

You will define data architecture, tooling, and standards, and play a founding role in shaping the data function as the company grows.