Senior Data Engineer: RAG Pipelines on Azure

RedCompass Labs; Greater London; On-site; GBP 70,000 - 90,000; Full time

Benefits offered by this job

Performance bonus

Health Insurance

Pension contributions

28 days annual holiday

Job summary

RedCompass Labs in Greater London is looking for a Senior Data Engineer to design and build data pipelines for payment systems. This role will involve working with regulatory documents to create a reliable data retrieval layer for AI applications.

The ideal candidate will have over 5 years of data engineering experience, strong Python skills, and expertise in Azure technologies. The job offers a performance bonus and comprehensive health benefits.

Qualifications

• 5+ years of experience as a data engineer with production pipelines.
• Experience with unstructured and semi-structured content processing.
• Proficient in Python and Azure data stack.

Responsibilities

• Design and build ingestion and processing pipelines for payments domain content.
• Engineer robust RAG pipelines tuned for technical content.
• Collaborate with team to translate domain content into retrieval layer.

Skills

Data pipeline design and operation

Python

Azure data stack knowledge

Data ingestion and processing

Embedding models

CI/CD with GitHub Actions

Tools

Azure Data Factory

Azure AI Search

Airflow

Docker

Job description

RedCompass Labs in Greater London is looking for a Senior Data Engineer to design and build data pipelines for payment systems. This role will involve working with regulatory documents to create a reliable data retrieval layer for AI applications.

The ideal candidate will have over 5 years of data engineering experience, strong Python skills, and expertise in Azure technologies. The job offers a performance bonus and comprehensive health benefits.