Source-linked AI summary
Order-Aware 2.5D Multiple Instance Learning for Preoperative MRI-Based Perineural Invasion Risk Assessment in Intrahepatic Cholangiocarcinoma
Go, Hyunsu, Han, Youngung, Kim, Kyeonghun, Jun, Jinyong, Lee, Junbeom, Kweon, Dohyun, Jeong, Yului, Park, Suah, Park, Sungha, Jung, Anna, Jeong, Woo Kyoung, Liao, Ken Ying-Kai, Lee, Hyuk-Jae, Kim, Nam-Joon
TL;DR
Preoperative PNI prediction in ICC is difficult because pathology provides postoperative patient-level labels without local MRI annotations. OAS-MIL represents tumor-centered T2-weighted MRI as ordered 2.5D slabs and combines set- and sequence-attention pooling. In five-fold patient-level cross-validation, it achieved a mean AUROC of 0.770 and outperformed evaluated volumetric and MIL baselines.
Problem
PNI is usually confirmed after resection, while preoperative MRI lacks slice- or voxel-level labels for weakly supervised prediction.
Method
OAS-MIL represents each tumor-centered MRI crop as an ordered sequence of contiguous 2.5D slabs and combines set-attention with bidirectional sequence-attention aggregation.
Results
0.770 mean AUROC was achieved in five-fold patient-level cross-validation, outperforming the evaluated volumetric and MIL baselines.
Takeaways & Limitations
The results suggest that axial order provides a useful inductive bias for weakly supervised PNI prediction from MRI in this cohort.
Abstract
from arXiv · showhide
Perineural invasion (PNI) is an adverse histopathologic marker in intrahepatic cholangiocarcinoma (ICC), but it is usually confirmed only after resection. Preoperative T2-weighted MRI may provide noninvasive imaging cues predictive of PNI, although labels are available only at the patient level without slice- or voxel-level annotations. We propose Order-Aware Slab Multiple Instance Learning (OAS-MIL), a weakly supervised framework for patient-level PNI prediction. Each tumor-centered MRI crop is represented as an ordered sequence of overlapping 2.5D slabs formed from contiguous axial slices. A shared encoder extracts slab-level features, which are aggregated by a permutation-invariant set-attention branch and a bidirectional sequence-attention branch. Using five-fold label-stratified cross-validation at the patient level, OAS-MIL achieved a mean AUROC of 0.770, outperforming the evaluated volumetric and MIL baselines. These results suggest that axial order provides a useful inductive bias for weakly supervised PNI prediction from MRI.
I. INTRODUCTION
OAS-MIL addresses weakly supervised preoperative PNI prediction in ICC, where patient-level labels lack local imaging annotations. It represents tumor-centered MRI as ordered 2.5D slabs and combines permutation-invariant and sequence-aware attention to use axial continuity.
- I. INTRODUCTION: PNI is clinically relevant in ICC because surgical studies associate it with recurrence and worse survival, yet it is usually established by postoperative pathology.Routine MRI lacks nerve-level or slice-level labels for model training.
- I. INTRODUCTION: Multiple instance learning fits this setting by supervising a bag of image instances with one patient-level label.Attention-based MIL provides differentiable bag-level pooling without local annotations.
- I. INTRODUCTION: Ordered 2.5D slabs retain local cross-slice information while avoiding the higher memory and optimization burden of fully volumetric encoders.Unlike exchangeable instances, axial slab order can encode anatomical continuity.
- I. INTRODUCTION: OAS-MIL models preoperative PNI prediction from tumor-centered T2-weighted MRI under patient-level supervision.The framework represents each crop as a complete ordered sequence of contiguous slabs.
- I. INTRODUCTION: The sequence-attention branch uses a bidirectional recurrent reader to model axial order alongside the existing set-attention branch.The evaluation uses patient-level, five-fold label-stratified cross-validation with ablations of representation and aggregation choices.
II. METHODS
The method represents each patient’s tumor-centered T2-weighted MRI crop as an ordered bag of overlapping axial slabs for patient-level PNI classification.
- Each patient is represented by a tumor-centered T2-weighted MRI crop X_i with a binary patient-level PNI label y_i.
- Crops are resampled to 1.0×1.0×5.0 mm spacing and standardized to a 192 × 192 × 160 mm field of view.This yields H = W = 192 and D = 32.
- Each axial position forms an odd-sized slab from contiguous slices, with boundary indices clamped to preserve one slab per position.
- The resulting patient bag retains the original through-plane order, with k = 3 used unless otherwise specified.
C. Shared Slab Encoder
A shared ResNet-18 encoder converts every MRI slab into a feature vector, using an input adaptation designed for variable slab channel counts.
- A shared adapted ResNet-18 backbone encodes every slab position into a feature vector of dimension d = 512.The same encoder is applied across all slab positions.
- The pretrained ResNet-18 first convolution is modified to accept the slab’s channel size through grayscale-based symmetric initialization.
- This initialization preserves symmetry across slab channels while keeping first-layer response scale comparable across slab sizes.
D. Set-Attention Branch
The set-attention branch pools slab features by computing normalized scalar attention scores, producing a permutation-invariant summary.
- The branch applies additive attention pooling over the slab bag and normalizes scalar scores across instances.
- Its output is a set-attention summary that aggregates the slab representations.
- Because the branch is permutation-invariant, its pooled representation does not depend on slab ordering.
E. Sequence-Attention Branch
The sequence-attention branch models slab features as an axial sequence, uses bidirectional recurrent context, and computes aggregation weights and a weighted summary from projected sequence states.
- E. Sequence-Attention Branch: The branch models slab features as an axial sequence using a one-layer bidirectional GRU to produce contextual states.
- E. Sequence-Attention Branch: Bidirectional states are concatenated and projected to the common feature dimension before attention scoring.
- E. Sequence-Attention Branch: The branch computes aggregation weights and a weighted summary from the projected sequence states.
- E. Sequence-Attention Branch: Attention scores and normalized through-plane weights are computed from the projected sequence states.
F. Prediction Head and Training Objective
The prediction head normalizes and concatenates the set- and sequence-attention summaries before producing predictions, while training uses weighted binary cross-entropy with logits.
- F. Prediction Head and Training Objective: Branch summaries are normalized with shared Layer-Norm, concatenated, and passed through a prediction head.
- F. Prediction Head and Training Objective: Training uses positive-class-weighted binary cross-entropy with logits, with the positive-class weight computed from the training split.
- F. Prediction Head and Training Objective: Dropout is applied during training in the feed-forward components.
A. Data and Evaluation Protocol
The study evaluated patient-level PNI prediction in a single-center MRI cohort using five-fold label-stratified cross-validation, with volumetric and ordered-slab MIL baselines compared by AUROC and AUPRC.
- A. Data and Evaluation Protocol: The cohort included 183 patients with preoperative T2-weighted MRI, comprising 70 PNI-positive and 113 PNI-negative cases.
- A. Data and Evaluation Protocol: Five-fold label-stratified cross-validation was performed at the patient level, with separate training, checkpoint-selection, and held-out testing folds.
- A. Data and Evaluation Protocol: Volumetric baselines used preprocessed crops, while MIL baselines used the same ordered 2.5D slab instances.
- A. Data and Evaluation Protocol: AUROC was the primary metric and AUPRC the complementary metric for the imbalanced endpoint.
- A. Data and Evaluation Protocol: OAS-MIL achieved AUROC 0.770 ± 0.077 and AUPRC 0.692 ± 0.093, versus CLAM-SB AUROC 0.705 ± 0.071 and AUPRC 0.608±0.118.
D. Ablation Analysis
Ablation and sensitivity analyses identified the strongest component configuration and examined computational profiling and robustness to crop-localization perturbations.
- D. Ablation Analysis: The 3-slice slab representation, fused set- and sequence-attention aggregation, and bidirectional GRU reader produced the strongest overall ablation results.
- D. Ablation Analysis: With test-time crop perturbations, OAS-MIL achieved AUROC 0.759 ± 0.069 and AUPRC 0.682±0.052, remaining close to unperturbed performance.
- D. Ablation Analysis: The perturbation analysis independently varied in-plane offsets from U(−10, 10) mm and through-plane offsets of −1, 0, or +1 axial slices without retraining.
- D. Ablation Analysis: The computational profile reports model size, multiply–accumulate operations, peak GPU memory, and forward time.
G. Qualitative Analysis
Figure 3 provides a qualitative audit of how set- and sequence-attention weight ordered slabs in a PNI-positive patient, alongside Grad-CAM and tumor-mask reference overlays. The visualization is interpretive rather than voxel-level localization.
- Aggregation weights concentrate over a limited slab interval near the tumor-containing region in the PNI-positive example.
- The bottom panel pairs the selected slab with a tumor-mask reference overlay and a Grad-CAM overlay showing a coarse response around the tumor and adjacent tissue.
- The top panel displays branch-wise aggregation-weight profiles across the ordered 2.5D slab montage.
- These outputs serve as qualitative audit cues rather than voxel-level PNI localization.