Data Engineer - End-to-End AI Search Pipelines (Equity)

United States Digital Space LLC; Greater London; On-site; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Visa sponsorship

Private medical insurance

Free meals

Job summary

Solve Intelligence is hiring a data engineer to build the ingestion and search systems behind our AI products in London. You’ll own pipelines from source data through to fast, scalable queries across large datasets, collaborating with AI researchers and product engineers.

You’ll work with Python, SQL and production databases to deliver reliable workflows for patent literature, case law, standards, and web content, ensuring data quality and traceability across sources.

Qualifications

• Proven experience designing and operating production databases.
• Experience building production data pipelines over large, messy datasets.
• Expertise with running search systems over large document collections.
• End-to-end ownership from raw data to user-facing functionality.
• Strong understanding of schema design, indexing and query optimisation.
• Experience diagnosing performance bottlenecks in live systems via profiling.

Responsibilities

• Large-scale ingestion: build and operate high-throughput pipelines with incremental updates and recovery.
• Document processing and data quality: extract content, handle malformed records and changing schemas.
• Search and serving: design schemas, indexes and partitioning for fast queries over millions of records.
• Connecting information across sources: link patents, scientific records and sources with traceability.
• Performance engineering: profile parsing, ingestion, and database builds for throughput and latency.

Skills

Python

SQL

Production databases

Data pipelines

Search systems

Schema design

Query optimisation

Performance profiling

End-to-end ownership

Tools

PostgreSQL/pgvector

OpenSearch/Elasticsearch

Spark/Delta Lake

AWS

NoSQL databases

Rust/C++

Job description

Solve Intelligence is hiring a data engineer to build the ingestion and search systems behind our AI products in London. You’ll own pipelines from source data through to fast, scalable queries across large datasets, collaborating with AI researchers and product engineers.

You’ll work with Python, SQL and production databases to deliver reliable workflows for patent literature, case law, standards, and web content, ensuring data quality and traceability across sources.