Source-linked AI summary
SGRNet: Spatially Guided Radiology Network for Structured Radiological Reporting of Head and Neck Cancer
Ayush Gupta, Vinkle Srivastav, Prateek Upadhya, Amit Gupta, Krithika Rangarajan, Nicolas Padoy
TL;DR
Dense head and neck anatomy, limited metabolic information, and hallucination risks motivate a structured alternative to free-text reporting from CECT. SGRNet combines anatomical and weak pathological spatial priors with feature modulation, achieving mAP 0.60 and an 8.8% relative improvement over raw 3D baselines.
Problem
Dense HNC anatomy, missing metabolic signaling in CECT, data scarcity, and hallucination risks make automated free-text reporting unsafe and difficult.
Method
The paper reformulates reporting as hierarchical anatomically grounded multi-label classification and uses organ segmentations plus weakly supervised 3D Gaussian tumor heatmaps for spatial feature modulation.
Results
mAP 0.60 is achieved across five densely packed subsites, an 8.8% relative improvement over corresponding raw 3D baselines.
Takeaways & Limitations
Structured prediction with localized spatial priors provides a factual-safety-oriented approach for HNC CECT reporting and improves performance on small, complex structures.
Takeaways & Limitations
The evaluation covers five subsites rather than the complete structured template, and the 145-case modeling set remains sensitive to rare atypical tumor morphologies.
Abstract
from arXiv · showhide
Automated radiological report generation can alleviate clinical workloads and eliminate observer variability. However, standard free-text generation models pose hallucination risks in dense regions and fail under data scarcity. We address these challenges in Head and Neck Cancer (HNC) from contrast-enhanced CT (CECT) imaging. To enforce factual safety, we reformulate report generation as an anatomically grounded, multi-label, structured reporting task, predicting localized tumor involvement across a hierarchical clinical schema. To bridge the visual gap from missing metabolic imaging (e.g., PET), we introduce SGRNet (Spatially Guided Radiology Network), incorporating two low-cost spatial priors: automated organ segmentations and weakly supervised tumor localization maps modeled via 3D Gaussian heatmaps. These priors are dynamically integrated via spatial feature modulation to guide the network toward subtle tumor-induced structural alterations. Evaluated on a multi-centric dataset of 184 paired HNC CECT volumes and reports, on five clinically salient, densely packed anatomical subsites, SGRNet achieves a mean Average Precision (mAP) of 0.60, an 8.8 percentage-point absolute improvement over strong volume-only 3D baselines.
1 Introduction
The paper targets the reporting burden and factual-safety challenges of head and neck cancer imaging, where dense anatomy and limited metabolic information complicate automation. It reformulates reporting as structured prediction and introduces SGRNet with spatial priors for CECT.
- Head and neck cancer reporting is burdensome because tightly packed subsites require an average of 24 minutes per scan.
- CECT is standard care but lacks direct metabolic signaling, forcing tumor localization to rely on subtle tissue attenuation changes.
- Unconstrained autoregressive VLMs pose safety risks because hallucinated text can radically alter staging.
- The study reformulates report generation as anatomically grounded multi-label prediction over a hierarchical schema and curates 184 paired HNC CECT scans and reports.
- SGRNet combines organ segmentations and weakly supervised 3D Gaussian tumor heatmaps through spatial feature modulation.
- An mAP of 0.60 represents an 8.8% relative improvement over baseline 3D architectures across five densely packed subsites.
2 Related Work
Prior radiology-reporting work has advanced image–text modeling mainly in 2D and thoracic 3D settings, while this paper focuses on anatomically constrained HNC CECT reporting. It combines structured prediction with spatial priors and weak supervision.
- Large paired image–text benchmarks have supported 2D free-text and structured prediction, but they do not extend to 3D volumetric relationships.
- Thoracic 3D report-generation models rely on CT-RATE and use unconstrained text decoders that pose hallucination risks for intricate head and neck workflows.
- This framework uses automated segmentation outputs as architectural constraints rather than as end goals for CECT report generation.
- Weak supervision uses coarse spatial annotations such as bounding boxes or extreme points to guide models without costly voxel-level labeling.
3 Dataset and Curation Pipeline
The dataset combines public and institutional HNC CECT data into a multi-centric cohort designed for structured radiological reporting. Its sources include 49 public scans and 135 institutional scans.
- The multi-centric cohort contains 184 contrast-enhanced CT scans from two distinct clinical streams.
- The public-domain stream contributes 49 ACRIN-HNSCC scans from TCIA after quality auditing for severe motion or artifacts.
- The institutional clinical stream contributes 135 consecutively collected contrast-enhanced CT scans from the All India Institute of Medical Sciences.
4 Method
SGRNet reformulates reporting as structured multi-label classification and processes CECT with anatomical and pathological spatial priors. These priors modulate visual features before fusion and prediction.
- 4 Method: SGRNet predicts an itemized anatomical schema using raw CECT volumes plus low-cost structural and pathological priors.
- 4 Method: The architecture has a primary imaging stream and two auxiliary streams that synthesize anatomical and pathological geometry.
- Automated Structural Priors (OrgSeg): TotalSegmentator extracts 21 HNC-relevant anatomical subsites, which are embedded and encoded into downsampled structural prior maps.
- Weak Pathological Priors via 3D Gaussian Heatmaps: A 3D nn-UNet trained on external PET/CT CT channels and fine-tuned on 39 box-annotated samples produces coarse tumor cuboids for weak localization.
- Weak Pathological Priors via 3D Gaussian Heatmaps: The cuboid dimensions determine directional Gaussian spreads, and a dedicated encoder converts the heatmap into pathological prior features.
- Spatial Feature Modulation: Spatial gating combines the prior maps into a focus map that modulates CECT features, amplifying subtle lesion signatures while preserving anatomical alignment.
- Unified Feature Fusion and Structured Prediction: The modulated CECT features are concatenated with raw prior maps, fused by a 3D convolutional network, pooled, and mapped to structured task probabilities.
5 Experiments & Results
Experiments compare volume-only CECT models, overlap-based priors, and learned prior-guided SGRNet variants on five anatomical subsites. SGRNet improves performance overall, while ablations indicate that spatial modulation and Gaussian tumor heatmaps are important contributors.
- 0.512 mAP for DenseNet121 and 0.494 mAP for EfficientNet-B0 were obtained by volume-only baselines, while CT-FM reached 0.256 AP on strap muscles.
- 0.600 mAP was achieved by the DenseNet121-based SGRNet, an 8.8% relative improvement over its raw-image baseline.
- +0.151 AP on hypopharynx and +0.184 AP on laryngeal airway were particularly pronounced SGRNet gains in densely packed structures.
- Spatial modulation consistently outperformed element-wise addition across both backbones in the prior-fusion ablations.
- 0.600 mAP was reached when the weak Gaussian tumor heatmap was added after anatomical organ maps in the DenseNet configuration.
- Table 1 reports Average Precision for baseline CECT models, the non-learned overlap baseline, prior-blending ablations, and final SGRNet variants.
6 Discussion & Limitations
The experiments support spatial priors as a corrective mechanism for low-contrast, densely packed anatomy, but performance and scope remain constrained. Limitations include motion-sensitive anatomy, small modeling data, imperfect tumor localization, segmentation-error propagation, and evaluation on only five subsites.
- Discussion: Explicit spatial boundaries and weak pathological context improve attention to tumor-adjacent boundaries in low-contrast 3D datasets.
- Discussion: 0.549 AP for baseline DenseNet121 exceeded 0.478 AP for the prior-guided framework on the mobile, deformable Tongue subsite.
- Limitations: 145 modeling cases leave the network sensitive to rare, highly atypical tumor morphologies.
- Limitations: Axis-aligned 3D cuboids can include healthy adjacent boundaries in Gaussian tumor distributions.
- Limitations: Automated TotalSegmentator errors can cascade into downstream feature modulation.
- Limitations: Evaluation covers five high-yield subsites rather than the complete structured template, leaving full-schema scaling for future work.
7 Conclusion
SGRNet is presented as a prior-guided 3D architecture for safe, structured reporting of head and neck cancer from CECT. The conclusion connects its spatial priors with improved performance over raw 3D baselines.
- SGRNet targets safe, structured radiological reporting of head and neck cancer on contrast-enhanced CT scans.
- The framework converts reports into deterministic structured subsite targets to avoid text hallucination risks from unconstrained vision-language models.
- 8.8% relative mAP improvement over raw 3D baselines is reported after dynamically modulating visual features with anatomical segmentations and weak 3D Gaussian heatmaps.
- The example report describes oropharyngeal involvement, cervical necrotic lymphadenopathy, and right-sided vessel encasement with internal jugular vein thrombosis.
- The structured reporting template is identified as an appendix resource for head and neck cancer CT examinations.
D Summary of Dataset Characteristics
The appendix documents dataset-level imaging characteristics and the anatomical label set used for segmentation. The label inventory covers airway structures, bones, muscles, glands, vessels, and tumor-related subsites.
- Table A.2 summarizes dataset-level imaging characteristics.
- Table A.3 lists segmented organs together with their corresponding label identifiers.
- The anatomical label inventory includes larynx air, thyroid and cricoid cartilage, hyoid bone, tongue, muscles, glands, arteries, veins, trachea, oropharynx, and hypopharynx.
F Segmentation Results
The appendix reports qualitative segmentation outputs for anatomical structures and tumors. The tumor results compare models trained on public CT data with a version further fine-tuned on manually curated cuboid regions.
- Qualitative organ-segmentation results show predicted masks for 21 anatomical structures across three representative patients.
- Tumor-segmentation results include an nnU-Net trained exclusively on CT images from a public PET/CT dataset.
- A second tumor-segmentation model was pre-trained on the public PET/CT dataset and fine-tuned using manually curated cuboid regions.
G Evaluation Metric
The evaluation section introduces the Dice similarity coefficient for comparing binary segmentation masks. The predicted and ground-truth masks are denoted by A and B, respectively.
- The Dice similarity coefficient is defined between two binary masks A and B.
- A and B denote the predicted and ground-truth masks, respectively.