Data Engineer: Build Scalable Pipelines for Public Archives

Digital Preservation Coalition; Greater London; On-site; GBP 55,000 - 75,000; Full time

Job summary

The National Archives (UK) in London invites applications for a Data Engineer to join the Digital Archiving team. You will help transform how archival data is managed, structured, and enriched to improve public access and discovery of the nation's documentary heritage.

You will work with a modern stack including Python, SQL, Shell scripting, Talend, XSLT, XML, CSV, JSON, RDF, and NoSQL databases to design scalable pipelines and automate data processing.

Qualifications

• Experience as a data engineer or similar role.
• Proficient in Python and SQL.
• Experience building scalable data pipelines.
• Familiar with XML/JSON/CSV data formats.
• Knowledge of graph databases is a plus.

Responsibilities

• Design and build scalable data pipelines for archival data.
• Automate data processing and enrichment.
• Develop tools for data integration across sources.

Skills

Python

SQL

Shell scripting

Data pipelines

XML

JSON

CSV

RDF

RDBMS

NoSQL

Graph databases

Talend

XSLT

Tools

Talend

XSLT

XML

CSV

JSON

RDF

RDBMS

NoSQL

Graph databases

Job description

The National Archives (UK) in London invites applications for a Data Engineer to join the Digital Archiving team. You will help transform how archival data is managed, structured, and enriched to improve public access and discovery of the nation's documentary heritage.

You will work with a modern stack including Python, SQL, Shell scripting, Talend, XSLT, XML, CSV, JSON, RDF, and NoSQL databases to design scalable pipelines and automate data processing.