GS-BrainText: A Multi-Site Brain Imaging Report Dataset from Generation Scotland for Clinical Natural Language Processing Development and Validation

arXiv cs.CL / 3/30/2026

📰 NewsSignals & Early TrendsIdeas & Deep AnalysisTools & Practical Usage

共有:

Key Points

The GS-BrainText dataset compiles 8,511 brain radiology reports from the Generation Scotland cohort, with 2,431 reports annotated for 24 brain disease phenotypes.
It is a multi-site UK dataset spanning five Scottish NHS health boards and includes a broad age distribution (mean 58, median 53), designed to support generalisable clinical NLP development and validation.
Expert annotations were produced using a defined schema with multidisciplinary clinical oversight, including 10–100% double annotation per site and formal quality assurance procedures.
Benchmarking with the rule-based EdIE-R system shows performance variability across health boards (F1 86.13–98.13), phenotypes (F1 22.22–100), and age groups (F1 87.01–98.13), underscoring generalisation challenges.
The release targets a gap in UK clinical text resources and enables research into linguistic variation, expression of diagnostic uncertainty, and how dataset characteristics affect NLP performance.

Abstract

We present GS-BrainText, a curated dataset of 8,511 brain radiology reports from the Generation Scotland cohort, of which 2,431 are annotated for 24 brain disease phenotypes. This multi-site dataset spans five Scottish NHS health boards and includes broad age representation (mean age 58, median age 53), making it uniquely valuable for developing and evaluating generalisable clinical natural language processing (NLP) algorithms and tools. Expert annotations were performed by a multidisciplinary clinical team using an annotation schema, with 10-100% double annotation per NHS health board and rigorous quality assurance. Benchmark evaluation using EdIE-R, an existing rule-based NLP system developed in conjunction with the annotation schema, revealed some performance variation across health boards (F1: 86.13-98.13), phenotypes (F1: 22.22-100) and age groups (F1: 87.01-98.13), highlighting critical challenges in generalisation of NLP tools. The GS-BrainText dataset addresses a significant gap in available UK clinical text resources and provides a valuable resource for the study of linguistic variation, diagnostic uncertainty expression and the impact of data characteristics on NLP system performance.