Data Engineer - SC Cleared

VIQU IT; City Of London; On-site; GBP 92,000 - 120,000; Full time

Job summary

Morela is recruiting a Data Engineer (RAG & Retrieval) for a 2-year contract in Central London, working on a UK government programme with an SC-cleared environment. You will own the data platform from ingestion to the retrieval layer powering AI capabilities, delivering robust pipelines and governance.

The role focuses on building RAG workflows, embeddings, and secure access controls, with hands-on experience in Python/SQL, cloud platforms and orchestration tools. Outside IR35 and rate TBC.

Qualifications

• Production data pipelines end-to-end across structured and unstructured data.
• Strong Python and SQL skills with data modelling, warehousing and lakehouse concepts.
• Experience with cloud data platforms (AWS/Azure/GCP) and containerised deployment (Docker, Kubernetes).
• Hands-on with orchestration and distributed processing (Airflow, Dagster, Spark).
• Experience with RAG, vector databases and retrieval pipelines.

Responsibilities

• Design, build and maintain robust batch and streaming ETL/ELT pipelines across sources.
• Model data for analytics with clear schemas, lineage and documentation.
• Own data quality, validation, monitoring and alerting for a trusted platform.
• Build RAG pipelines: embeddings, hybrid search, reranking, context assembly.
• Enforce document-level access controls and security in retrieval layers.
• Work with self-hosted or open-weight models in secure environments.

Skills

Python

SQL

Data modelling

Warehousing

Lakehouse

Airflow

Dagster

Spark

Cloud platforms

Containerisation

Tools

Kafka

OpenSearch

Qdrant

Weaviate

LangChain

Vector embeddings

Job description

MORELA

AI & Engineering Recruitment

Data Engineer (RAG & Retrieval)

2-year contract Central London Active SC Outside IR35

Morela is supporting a specialist technology consultancy on a 2-year Data Engineer contract in Central London, delivering on a UK government programme. Outside IR35, rate TBC. We need a genuinely strong all-round data engineer first and foremost, with recent hands-on RAG experience on top. Active SC is essential - eligibility will not be considered.

About the Client

A specialist technology consultancy embedding engineers into SC-cleared government delivery teams. They focus on long-term programme work and give their engineers real ownership of what they build.

The Role

You will own the data platform behind the programme, from ingestion and modelling through to the retrieval layer that powers its AI capability. The core of the job is solid data engineering - reliable pipelines, clean models and well-governed data. On top of that you will build the RAG pipelines that turn large volumes of unstructured material into accurate, access-controlled answers.

Responsibilities

Core data engineering

• Design, build and maintain robust batch and streaming pipelines (ETL/ELT) across structured and unstructured sources
• Model data for analytics and downstream use, with clear schemas, lineage and documentation
• Own data quality, validation, monitoring and alerting so the platform can be trusted
• Optimise pipelines and storage for performance, cost and scale
• Apply security, governance and access controls appropriate to an SC environment

RAG and retrieval

• Build ingestion for unstructured content, including document parsing, OCR and metadata enrichment
• Build production RAG pipelines: chunking, embeddings, hybrid search (vector + BM25), reranking and context assembly
• Enforce document-level access controls at query time
• Evaluate retrieval and generation quality with measurable metrics, and iterate
• Work with self-hosted and open-weight models in secure or air-gapped environments
• SC clearance: Active SC held, not eligibility

Essential Requirements

Core data engineering

• Strong commercial data engineering background, building and running production pipelines end to end
• Excellent Python and SQL
• Solid grasp of data modelling, warehousing and lakehouse concepts
• Hands-on with orchestration (e.g. Airflow, Dagster) and distributed processing (e.g. Spark)
• Experience with cloud data platforms (AWS, Azure or GCP) and containerised deployment (Docker, Kubernetes)
• Good engineering practice: version control, CI/CD, testing and infrastructure as code

RAG

• Recent RAG delivery: Hands-on RAG systems built and shipped recently - current work, not an old proof of concept
• Vector databases (e.g. pgvector, OpenSearch, Qdrant, Weaviate), embeddings, hybrid search and reranking
• Experience evaluating RAG systems properly (e.g. faithfulness)
• Streaming platforms such as Kafka
• Agentic or multi-hop retrieval, GraphRAG, query rewriting
• vLLM or similar for on-prem model serving
• LangChain, LlamaIndex, RAGAS or Langfuse
• Government, defence or national security programmes

Working Environment

Central London, SC-cleared environment, embedded in the programme team. Outside IR35, 2-year contract.