Source-linked AI summary
ASTAR: Automated induction of STAndardized radiology Reporting templates from large-scale clinical free-text corpora
Xinfeng Zhang, Mingxuan Liu, Yifei Chen, Juncheng Zhu, Kasidit Anmahapong, Yiming Huang, Yuan Zhang, Hongjia Yang, Yi Liao, Gang Ning, Haibo Qu, Qiyuan Tian
TL;DR
Radiology template construction remains a manual, static bottleneck despite advances in LLM-based information extraction, limiting scalable structured reporting. ASTAR automatically induces templates from large-scale free-text corpora through concept-slot extraction, hierarchical organization, and refinement. On 4,215 multi-center fetal brain MRI reports, it outperformed two expert-designed templates across coverage, fidelity, and usability while reducing development from weeks to hours.
Problem
Manual expert-consensus template construction is labor-intensive, difficult to scale, and may miss the linguistic and clinical diversity of large radiology-report corpora.
Method
ASTAR uses an LLM-based three-stage framework that constructs concept slots, organizes them into a hierarchical template tree, and refines the resulting structure.
Results
ASTAR outperformed two expert-designed templates across template quality, structuring fidelity, and radiologist-rated usability on 4,215 multi-center fetal brain MRI reports.
Takeaways & Limitations
ASTAR provides a scalable, data-driven complement to labor-intensive expert consensus for standardized radiology reporting template construction.
Abstract
from arXiv · showhide
Structured reporting converts free-text radiology narratives into queryable data keys, facilitating cohort assembly, longitudinal tracking, and training label generation for medical AI. The prevailing paradigm follows a two-stage pipeline: (1) constructing a reporting template, (2) extracting information to populate it. While the extraction stage has benefited from advances in large language models (LLMs), template construction remains a manual bottleneck relying on labor-intensive expert consensus that is static, difficult to scale, and may fail to capture real-world reporting diversity. We address this limitation with \textbf{\texttt{ASTAR}}, an LLM-based framework for Automated induction of STAndardized radiology Reporting templates from large-scale clinical free-text corpora. Extensive experiments on 4,215 fetal brain MRI reports from multiple centers demonstrate that the \textbf{\texttt{ASTAR}}-induced template surpasses two expert-curated templates across template coverage, information fidelity, diagnostic fidelity, and expert-rated usability, reducing template development from weeks of committee deliberation to hours of automated processing. Code: https://github.com/birthlab/ASTAR
1 Introduction
Radiology reports contain valuable clinical information, but free-text variability limits systematic reuse. Structured reporting addresses this problem, while ASTAR automates template induction to overcome the manual bottleneck in template construction.
- Radiology reports combine categorical descriptors, measurements, and diagnostic impressions but remain difficult to query because findings are stored as heterogeneous free text.Variability occurs across radiologists, institutions, and languages.
- This heterogeneity hinders case retrieval, cohort assembly, longitudinal tracking, and medical-AI label generation.
- Structured reporting converts narratives into queryable keys that support rapid cohort identification and ground-truth generation.
- The prevailing workflow constructs a reporting template and then extracts information from reports to populate it.LLMs and NLP have primarily advanced the extraction stage.
- Template construction remains a labor-intensive expert-consensus bottleneck, producing static templates that may miss linguistic diversity and evolving clinical descriptions.
- ASTAR automatically mines unified reporting structures from large-scale corpora and was evaluated on 4,215 fetal brain MRI reports from three centers.The authors describe it as a scalable, data-driven complement to expert consensus.
2 Method
ASTAR induces a hierarchical radiology template through concept-slot construction, tree organization, and refinement, then evaluates coverage, fidelity, and clinician usability. The protocol measures both preservation of report information and diagnostic content on held-out data.
- Method: ASTAR represents a template as a hierarchical tree whose leaf fields specify standardized names, value types, and type-dependent value specifications.
- Stage I: Concept Slots Library Construction: Stage I extracts and deduplicates clinical spans, embeds them, clusters them, and uses LLM subpartitioning to induce canonical concept slots.K-means cluster count is selected by maximizing silhouette score over C ∈ [50, 300].
- Stage II: Hierarchical Template Tree Construction: Stage II organizes slots into a clinically meaningful hierarchy through incremental assignment followed by global path refinement.Slots may be attached to existing nodes, used to propose branches, or dropped.
- Stage III: Template Refinement: Stage III consolidates synonymous values, harmonizes sibling value types, and standardizes redundant concepts across tree branches.Examples include mapping synonymous normality descriptions to “Normal” and unifying anatomical terminology.
- Evaluation Protocol: Evaluation uses template quality, structuring fidelity, and radiologist judgment to assess coverage, information preservation, diagnostic fidelity, and usability.The protocol was introduced because no widely used radiology-template evaluation protocol exists.
- Evaluation Protocol: Template quality measures case-level and key-level coverage on held-out reports, using LLM judgments of semantic matches between report keys and template keys.
3 Experiments and Results
Experiments used multi-center fetal brain MRI reports to compare ASTAR with two expert-designed templates and evaluate scaling behavior. ASTAR achieved stronger coverage and fidelity, while larger corpora mainly improved rare-case coverage rather than average fidelity.
- Data Collection: 4,100 reports were collected, with 4,000 used for ASTAR template construction, 100 held out for ID testing, and 115 additional reports used for OoD testing.The data spanned examinations from October 2015 to December 2024 across multiple MRI systems and institutions.
- Comparison with Expert-Designed Templates: ASTAR was compared with the ESPR Template and the FetalExtract Template, which were both expert-designed baselines.The ESPR template came from a European Society of Paediatric Radiology task force, while FetalExtract was compiled from ESPR guidelines, MeSH, and a medical dictionary.
- Comparison with Expert-Designed Templates: 81.34%/82.42% case-/key-level coverage was achieved by ASTAR on the ID dataset, with higher coverage than both expert-designed templates.The reported coverage advantage persisted on the OoD set, although the gap narrowed under domain shift.
- Comparison with Expert-Designed Templates: 0.694±0.100 ROUGE-L was achieved by ASTAR on ID reports, while OoD ROUGE-L remained higher at 0.503 ± 0.192 than ESPR and FetalExtract.On ID, ASTAR also achieved BERTScoreR of 0.895±0.016 versus 0.892±0.016 and 0.882±0.021.
- Scaling Behavior of ASTAR: 39.7% to 14.5% was the decrease in the case-level coverage tail ratio as corpus size increased from n=32 to n=2048, before stabilization.Key-level coverage remained 0.767–0.859, while mean ROUGE-L stayed 0.676–0.696 and the report-level tail ratio fell to 3.1% at n=4000.
4 Conclusion
ASTAR automatically induces standardized radiology reporting templates from large-scale free-text corpora. On 4,215 multi-center fetal brain MRI reports, it outperformed two expert-designed templates across template quality, structuring fidelity, and radiologist-rated usability while reducing development from weeks to hours.
- Conclusion: ASTAR outperformed two expert-designed baseline templates across template quality, structuring fidelity, and radiologist-rated usability on 4,215 multi-center fetal brain MRI reports.Template development was reduced from weeks of expert consensus to hours of automated processing.