ESRC Trusted AI Project Post-Doctoral Researcher

Northumbria University; Newcastle upon Tyne; On-site; GBP 34,000 - 52,000; Full time

Job summary

Northumbria University invites applications for a postdoctoral Research Associate to lead data-workflow design for the Trustworthy AI for Peer Review project. You will build REF-aligned corpora, integrate full-text and metadata pipelines, and ensure reproducible research practices.

Based in Newcastle upon Tyne and London, you will collaborate with colleagues across a multidisciplinary team, publish results, present findings, and help shape demonstrators and policy-relevant outputs for research

Qualifications

• PhD in a relevant field with postdoctoral experience preferred.
• Evidence of data handling and reproducible workflows.
• Experience with NLP, LLMs, bibliometrics or scientometrics is desirable.

Responsibilities

• Develop reproducible data workflows linking REF submissions and metadata.
• Link and curate datasets from sources like OpenAlex, Crossref, Unpaywall and REF data.
• Contribute to LLM evaluation tasks and model comparison experiments.
• Publish and present findings; support policy-relevant demonstrators.

Skills

Data handling

Computational text analysis

Reproducible research

Python

APIs

Web data extraction

Text processing

Version control

Education

PhD in a relevant field

Tools

Python

APIs

OpenAlex

Crossref

Unpaywall

REF data

Job description

ABOUT THE ROLE

As a Research Associate, you will take a leading role in the project Trustworthy AI for Peer Review: Developing and Validating Composite Research Quality Indicators. Your work will focus on constructing high-quality REF-aligned corpora of research outputs, using and extending the full-text cache and metadata pipelines developed through our software: Macroscope. You will collect, clean, link, and document text and bibliographic data so that it can support rigorous testing of AI-assisted research assessment.

Job Description

You will be responsible for developing reproducible data workflows that link REF2021 submissions, Units of Assessment, outcome profiles, institutional metadata, bibliographic records, abstracts, and, where licensing permits, full-text content. This will involve working with sources such as OpenAlex, Crossref, Unpaywall, Scopus and REF data, designing robust extraction and matching methods, maintaining provenance and quality-control records, and producing well-structured datasets for downstream analysis.

The role will also contribute to the project’s LLM-based evaluation work, helping to design, run, and analyse experiments that test how large language models perform in REF-style assessment tasks. This will include supporting prompt and model comparison, repeated-run evaluation, assessment of stability and disagreement, and analysis of potential biases or confounding effects. The post provides an opportunity to work at the frontier of trustworthy AI, research evaluation, scientometrics, and large-scale text infrastructure, with scope to publish, present, and help shape policy-relevant demonstrators.

The successful candidate will also have the opportunity to support in the development of Semantic Space, an Academic Intelligence company that supports strategic decision making for research intensive organisations.

The role is fixed term for 12 months.

About The Project

Trustworthy AI for Peer Review: Developing and Validating Composite Research Quality Indicators is a UKRI/ESRC Metascience project investigating how artificial intelligence can support high-stakes research assessment without displacing expert judgement. Research assessment through peer review, journal editorial processes and the Research Excellence Framework (REF) is essential to the research system, but it is increasingly costly, time-consuming and difficult to scale. At the same time, large language models are creating new possibilities for analysing and evaluating research outputs.

The project addresses a central challenge: AI-derived research indicators are often presented as single scores, with limited visibility of uncertainty, disagreement, instability or bias. The project will develop and validate new approaches that make the reliability of AI-augmented assessment explicit and usable by decision-makers. It will combine benchmark datasets, LLM evaluation, scientometric indicators, uncertainty-aware modelling, Trust Cards, and user-centred demonstrators for REF panels, journal editors, publishers and research managers.

Key Research Areas Include

• Building FAIR, multi-resolution benchmark datasets linking REF submissions, journal peer-review data, books, bibliographic metadata, abstracts and, where appropriate, full-text corpora.
• Developing and testing LLM-based approaches to REF-style and peer-review assessment, including repeated-run evaluation, prompt comparison, model comparison, and calibration against known assessment outcomes.
• Creating indicators that communicate uncertainty, disagreement, stability, bias sensitivity and limits of valid use, rather than reducing complex judgements to opaque single scores.
• Developing transparent governance and communication tools, including Trust Cards, open-source workflows, documentation and demonstrators for responsible use of AI in research evaluation.

The project is delivered by an interdisciplinary consortium led by Loughborough University with Northumbria University, Durham University, Heriot-Watt University, the University of Southampton, the University of Aberdeen and the University of Wolverhampton, bringing together expertise in machine learning, scientometrics, accountable AI, computational reasoning, human-centred design, metascience and research evaluation practice.

About The Team

The project team brings together researchers working across trustworthy AI, machine learning, scientometrics, human-centred design, computational reasoning, research policy and responsible innovation. The wider consortium includes specialists in uncertainty-aware AI, peer-review analysis, accountable AI, argumentation, stakeholder engagement and research evaluation.

At Northumbria University, the role will be based within Professor Martyn Dade-Robertson’s research environment, building on existing work on Macroscope: a software and data pipeline for mapping, analysing and interpreting large corpora of academic research. The Northumbria contribution focuses on the empirical data backbone of the project, including corpus construction, full-text and metadata processing, REF-aligned benchmarking, and interpretable demonstrators that help experts understand complex research landscapes (see www.semanticspace.ai ).

About You

We are looking for a highly motivated postdoctoral Research Associate with strong skills in data handling, computational text analysis and reproducible research. You will have experience of programming for data-intensive research, ideally using Python and associated tools for APIs, web data extraction, text processing, databases, data cleaning, version control and workflow documentation. Experience with NLP, large language models, bibliometrics, scientometrics, research information systems, OpenAlex/Crossref/Unpaywall, REF data, or full-text extraction pipelines would be especially valuable. You do not need prior experience of REF assessment, but you should be interested in how data and AI can be used responsibly in high-stakes research evaluation.

Further information about the requirements of the role is available in the person specification.

If you would like an informal discussion about the role, please contact Prof Martyn Dade-Robertson ([e-mail]).

About Us

Join Northumbria University, a research-intensive institution unlocking potential and changing lives locally and globally. Named Times Higher Education's University of the Year in 2022 and Modern University of the Year in 2025, we rank top 25 in the UK for research power. Discover more about us .

With over 37,000 students from 140+ countries, we offer world-leading research, award-winning partnerships, and an outstanding student experience. We empower our exceptional staff, promoting a positive work-life balance and offering great benefits, including excellent pension schemes, flexible working, generous holiday entitlement and more .

Our Northumbria Values, co-created by our team, define who we are: Academic Excellence, Innovation, Inclusivity, Collaboration, and Ambition. Our Behaviours shape our work culture: We listen and learn, support one another, respect everyone, trust each other, and are bold.

Based in Newcastle upon Tyne and London, we are an on-campus organisation and offer flexible hours and location where the role allows. We pride ourselves on diversity and inclusivity, holding numerous awards for gender and race equality, disability confidence, and research excellence. We also hold the HR Excellence in Research award for implementing the concordat supporting the career Development of Researchers and are members of the Euraxess initiative to deliver information and support to professional researchers. The University has implemented a range of flexible working arrangements, and we are happy to explore candidate requirements as part of the recruitment process.

Explore " Our Exceptional People , Our Values and Behaviours " page to learn more about our commitments and see our Privacy Policy for further details