Source-linked AI summary
CNsEMD: An Expert-Annotated Multi-Field-Strength MRI Dataset and a Hyperspherical Manifold Network for Multimodal Cranial Nerve Parcellation
Lei Xie, Junxiong Huang, Guoqiang Xie, Jiawei Zhang, Jiahao Huang, Tianling Lyu, Ye Wu, Mingchu Li, Shoujun Yu, Shanshan Wang, Qingrun Zeng, Yuanjing Feng
TL;DR
Accurate cranial-nerve parcellation is difficult, and structural MRI alone lacks sufficient contrast and fiber-direction information. This paper introduces a public multi-field-strength expert-annotated dataset and PHM-Net, which performs multimodal interaction in a shared hyperspherical space. The authors report best average performance among compared methods under heterogeneous imaging conditions.
Problem
Structural images alone provide insufficient contrast and lack directional information about fiber orientation for accurate CN parcellation.
Source: §I · p. 2Method
PHM-Net combines sign-invariant magnitude-preserving DEC encoding, angular cross-modal interaction, and prototype-based voxel classification in a shared hyperspherical space.
Source: §I · p. 2Results
PHM-Net achieves the best average performance among compared parcellation and multimodal learning methods under heterogeneous imaging conditions.
Source: §V · p. 10Takeaways & Limitations
CNsEMD provides a reproducible benchmark, and PHM-Net is a geometry-consistent solution for multimodal CN parcellation under heterogeneous MRI acquisition.
Source: §V · p. 10
Abstract
from arXiv · showhide
Cranial nerves (CNs) play essential roles in sensory, motor, and autonomic functions. Accurate CN parcellation from multimodal magnetic resonance imaging (MRI) is crucial for neuroanatomical analysis and neurosurgical planning. However, accurate CN parcellation remains extremely challenging because CNs are very small, exhibit low image contrast, and have slender tubular morphologies and complex anatomical trajectories. Moreover, the lack of publicly available, expert-annotated datasets has impeded the development and fair benchmarking of learning-based CN analysis methods. In this work, we introduce CNsEMD, an expert-annotated multimodal dataset for CN parcellation. It comprises data from 202 subjects acquired on 3T, 5T, and 7T MRI scanners. We further propose the projective hyperspherical manifold network (PHM-Net), which learns cross-modal representations by capturing angular relationships in a shared hyperspherical embedding space. Rather than performing multimodal fusion in Euclidean space, the proposed Hyperspherical cross-modal interaction (HCI) module enables bidirectional feature exchange between T1-weighted (T1w) and direction-encoded color (DEC) representations on a unit hypersphere. The Magnitude-preserving projective hyperspherical orientation representation (PHOR) captures the axial nature of DEC orientations while preserving diffusion magnitude. The hyperspherical prototype segmentation head (HPSH) further extends angular similarity to voxel-wise classification using normalized voxel embeddings and learnable class prototypes. Extensive experimental results on the CNsEMD dataset demonstrate the effectiveness of our PHM-Net against state-of-the-art methods. CNsEMD establishes a reproducible benchmark for multimodal CN imaging, while PHM-Net provides a geometry-consistent solution for CN parcellation across diverse MRI acquisitions.
I. INTRODUCTION
CN parcellation is difficult because cranial nerves are small, low-contrast, slender, and anatomically complex, while expert-annotated multimodal benchmarks have been unavailable. The paper introduces CNsEMD and PHM-Net, which performs multimodal interaction in a shared hyperspherical space and reports best overall performance against 12 state-of-the-art models.
I. INTRODUCTION
- CN parcellation remains challenging because nerves are small, low-contrast, slender, and follow complex anatomical trajectories.Specialists integrate complementary multimodal MRI information when identifying CNs.Source: §I · p. 1
- Structural-only parcellation lacks sufficient contrast and fiber-orientation information, while existing fusion often treats DEC as generic features and operates in Euclidean space.Hyperspherical learning instead compares normalized features by angle, motivating the paper's cross-modal interaction design.Source: §I · p. 2
- Prior multimodal CN datasets lacked public expert annotations, limiting fair and reproducible evaluation across heterogeneous acquisition settings.Existing datasets relied predominantly on private cohorts and varied in sources, preprocessing, modalities, and annotation criteria.Source: §I · p. 2
- PHM-Net combines bidirectional angular feature exchange through HCI, sign-invariant magnitude-preserving DEC encoding through PHOR, and prototype-based voxel classification through HPSH.The network learns cross-modal representations in a shared hyperspherical embedding space.Source: §I · p. 2
- CNsEMD is an expert-annotated multimodal dataset comprising 202 subjects scanned at 3T, 5T, and 7T and covering CN II, III, V, and VII/VIII.The dataset and code are publicly available.Source: §I · p. 2
- PHM-Net consistently achieves the best overall performance against 12 state-of-the-art general, multimodal, and CN-specific models on CNsEMD.The experiments are described as demonstrating superior accuracy and robustness across heterogeneous imaging conditions.Source: §I · p. 2
II. CNSEMD DATASET
CNsEMD combines aligned structural and diffusion MRI with voxel-level expert annotations from 202 subjects acquired at three field strengths. Its annotations undergo staged model-assisted refinement and expert review, while PHM-Net processes the modalities through a dual-branch architecture with hyperspherical interaction and prototype-based voxel classification.
A. Dataset Acquisition
- CNsEMD includes 202 subjects acquired at 3T, 5T, and 7T, with aligned T1w images, DEC images, and expert-curated voxel-level annotations.The labels cover CN II, III, V, VII, and VIII, with VII and VIII treated as one foreground category, yielding four foreground classes.Source: A. Dataset Acquisition · p. 3
B. Manual Annotation Protocol
- Three medical postgraduate annotators refined candidate masks slice by slice, followed by review from two CN-anatomy experts and quality assessment by a senior neurosurgeon.Cases failing predefined criteria were iteratively corrected until consensus; voxel-level annotations were obtained for all 202 subjects.Source: B. Manual Annotation Protocol · p. 3
A. Overview of PHM-Net
- PHM-Net encodes T1w and sign-invariant, magnitude-preserving DEC representations in separate branches, exchanges features through angular HCI, and uses HPSH for voxel-wise parcellation.HCI operates at the last three encoder stages; HPSH compares normalized decoder embeddings with learnable class prototypes.Source: A. Overview of PHM-Net · p. 4, Figure 3 · p. 4
B. Magnitude-Preserving Projective Hyperspherical Orientation Representation
PHOR encodes DEC orientations to account for their antipodal sign ambiguity while retaining diffusion magnitude. Its projective representation converts each voxel into a seven-channel representation before the DEC encoder.
B. Magnitude-Preserving Projective Hyperspherical Orientation Representation
- In the diffusion tensor model, the eigenvector v1 associated with the largest eigenvalue λ1 represents principal diffusion orientation, while FA quantifies diffusion anisotropy.A DEC map combines these quantities into a voxel-wise vector.Source: B. Magnitude-Preserving Projective… · p. 4
- PHOR addresses the sign ambiguity of DEC orientations while preserving their FA-related magnitude, which signed components or orientation-only normalization would not jointly retain.The same physical orientation is represented by v1 and −v1, while normalizing the vector alone would discard FA-related magnitude.Source: B. Magnitude-Preserving Projective… · p. 4
- The normalized orientation is embedded through the six independent components of ddT, making the representation invariant to sign reversal and identifying antipodal directions.The orientation axis is represented in S2/{d ∼−d}, isomorphic to the real projective plane RP2.Source: B. Magnitude-Preserving Projective… · pp. 4–5
- PHOR combines a magnitude channel with six projective orientation channels scaled by that magnitude, producing a parameter-free seven-channel representation independently at each voxel.The first channel retains m, and the six orientation channels preserve their association with diffusion anisotropy.Source: B. Magnitude-Preserving Projective… · p. 5
C. Hyperspherical Cross-modal Interaction (HCI)
HCI exchanges information between T1w and DEC features using angular relationships in a shared hyperspherical space. It interacts at encoder stages 3–5 and passes refined features to the decoder.
C. Hyperspherical Cross-modal Interaction (HCI)
- HCI removes scale dependence before measuring correspondence between T1w anatomy features and DEC orientation-and-strength features, while preserving symmetric information flow between modalities.Their channel magnitudes are not directly comparable in Euclidean feature space.Source: C. Hyperspherical Cross-modal… · p. 5
- At encoder stages 3–5, modality-specific projections map T1w and DEC features to a common dimension before their spatial dimensions are flattened into tokens.The projected tokens are ℓ2-normalized along the embedding dimension.Source: C. Hyperspherical Cross-modal… · p. 5
- HCI computes cross-modal angular affinity between normalized tokens and separately normalizes interaction weights in each direction, allowing each modality to retrieve complementary information.The temperature controls weight concentration: smaller values emphasize the most angularly consistent correspondences, while larger values yield more diffuse aggregation.Source: C. Hyperspherical Cross-modal… · p. 5
- After residual aggregation, interacted features are renormalized, reshaped, and reprojected; they feed subsequent encoder stages and decoder skip connections at multiple semantic scales.Interaction follows progressively larger receptive fields at stages 3–5, leaving early high-resolution encoder stages unchanged.Source: C. Hyperspherical Cross-modal… · p. 5
D. Hyperspherical Prototype Segmentation Head (HPSH)
HPSH carries angular geometry into voxel-wise classification by comparing normalized decoder embeddings with learnable class prototypes. Its class logits aggregate prototype similarities.
D. Hyperspherical Prototype Segmentation Head (HPSH)
- HPSH classifies voxels by cosine similarity between normalized decoder embeddings and learnable class prototypes, rather than unconstrained Euclidean inner products.Each prototype represents a trainable class direction on the unit hypersphere, so predictions depend on angular compatibility rather than feature or classifier-weight magnitudes.Source: D. Hyperspherical Prototype… · p. 5
- The embedding function maps the final decoder's fused multimodal feature map to a Dp-dimensional space, where voxel embeddings are normalized before classification.The embedding function uses a 3 × 3 convolution followed by batch normalization and ReLU, then a 1 × 1 convolution; Dp = 32.Source: D. Hyperspherical Prototype… · pp. 5–6
- HPSH aggregates prototype similarities into class logits with log-mean-exp, and the final model uses three prototypes per class to capture heterogeneous voxel embeddings.Angular compatibility remains the basis of the final voxel-wise decision.Source: D. Hyperspherical Prototype… · p. 6
E. Training Objective
PHM-Net is trained end to end with cross-entropy and soft Dice losses. The Dice term is computed over foreground classes to address severe background-to-CN class imbalance.
E. Training Objective
- The end-to-end training objective combines cross-entropy and soft Dice losses, with λCE and λDice controlling their relative contributions.The setup defines C = 5 classes including background.Source: E. Training Objective · p. 6
- Soft Dice is computed over the four foreground classes to mitigate severe imbalance between background and small cranial nerve structures.The loss uses one-hot class indicators and ε = 10^-5 for numerical stability.Source: E. Training Objective · p. 6
A. Experimental Settings
The experiments use subject-level five-fold validation on 162 development subjects and reserve 40 subjects for testing. All methods share the same partitions, preprocessing, training schedule, and hardware.
A. Experimental Settings
- 40 subjects (20 3T, 10 5T, 10 7T) were reserved solely for testing, while the other 162 subjects were split into five folds.Only slices containing annotated CN voxels were used, and the five fold-specific models were ensembled for test inference.Source: A. Experimental Settings · p. 6
- Training used paired one-channel T1w and three-channel DEC slices, with no data augmentation during training or validation.T1w slices were percentile-clipped and z-score normalized; non-finite DEC values were set to zero and finite values left unchanged.Source: A. Experimental Settings · p. 6
- All methods used the same subject-level partitions, preprocessing, training schedule, and hardware, with outputs covering background and four foreground CN classes.Methods were evaluated using Dice, Jac, ASD, and AHD, averaged over four foreground classes and reported as mean ± standard deviation across subjects.Source: A. Experimental Settings · p. 6
B. Comparison with State-of-the-Art Methods
PHM-Net is compared with 12 representative methods on a mixed-field-strength test cohort and separate 3T, 5T, and 7T subsets. It leads the mixed cohort on all four metrics and shows stronger overlap performance across field strengths, with qualitative examples illustrating fine-structure delineation.
B. Comparison with State-of-the-Art Methods
- The comparison covered 12 methods spanning general-purpose parcellation, multimodal fusion, and CN-specific approaches.CNTSeg-v2 was evaluated with T1w and DEC as its full input set, without modality dropping.Source: B. Comparison with State-of-the-Art… · pp. 6, 8
- PHM-Net led all four metrics on the mixed 40-subject test cohort: Dice 76.58%, Jac 62.92%, ASD 0.453 mm, and AHD 0.358 mm.Against CNTSeg-v2, Dice and Jac rose by 0.72 and 0.85 percentage points, while ASD and AHD fell by 0.083 and 0.074 mm.Source: Table II · p. 7, B. Comparison with State-of-the-Art… · p. 8
- Across 3T, 5T, and 7T subsets, PHM-Net achieved the best Dice and Jac; it led all four metrics at 5T but ranked second on 3T surface distances.Several competing methods had lower surface distances at 7T.Source: B. Comparison with State-of-the-Art… · p. 8
- Representative cases showed more complete 3D reconstructions and fewer false positives, including closer CN V cisternal-region agreement and consistent CN II optic-tract delineation.Fig. 4 presents 3T, 5T, and 7T cases with 3D reconstructions, slice-wise results, and enlarged challenging regions; colored overlays show expert references and red contours show predictions.Source: Figure 4 · p. 7, B. Comparison with State-of-the-Art… · p. 8
C. Ablation Analysis
The ablations evaluate PHM-Net components on CN parcellation, with variants selected on the development cohort before held-out testing. The complete model achieved the strongest performance across the four reported metrics.
C. Ablation Analysis
- PHOR, HCI, and HPSH were predefined before test evaluation, with hyperparameters selected on the development cohort; ablation variants were then evaluated on held-out subjects.The ablation study reports component effects on CN parcellation performance.Source: C. Ablation Analysis · p. 8, Table III · p. 8
1) Ablation of HCI:
- HCI raised Dice from 75.63% to 75.87% and Jac from 61.74% to 62.05%, while reducing ASD from 0.733 to 0.531 mm and AHD from 0.608 to 0.430 mm.The feature analysis found increased within-modality compactness and stronger spatially paired cross-modal alignment across all four CN categories.Source: 1) Ablation of HCI · p. 8, Table III · p. 8, Figure 5 · p. 8
2) Ablation of PHOR and HPSH:
- Adding PHOR to the Euclidean baseline raised Dice from 75.63% to 76.15% and Jac from 61.74% to 62.36%, while lowering ASD from 0.733 to 0.488 mm and AHD from 0.608 to 0.391 mm.These results support sign-invariant DEC orientation encoding.Source: 2) Ablation of PHOR and HPSH · p. 8
- Replacing the conventional 1 × 1 classifier with HPSH yielded the complete model’s best results: Dice 76.58%, Jac 62.92%, ASD 0.453 mm, and AHD 0.358 mm.The preceding HCI model had Dice 76.08% and Jac 62.26%, but higher ASD and AHD of 0.595 and 0.489 mm.Source: 2) Ablation of PHOR and HPSH · p. 9
D. Parameter Sensitivity Analysis
The section reports parameter choices, paired comparisons, computational cost, and input-modality results, alongside the paper’s overall conclusion. PHM-Net is presented as a geometry-consistent approach evaluated on a multi-field-strength dataset.
D. Parameter Sensitivity Analysis
- Dh = 512 was selected for its lowest ASD and AHD with comparable overlap, while K = 3 prototypes per class gave the highest Dice and Jac.Dice and Jac were relatively stable across tested projection dimensions; K = 2 and K = 4 had weaker overall results.Source: D. Parameter Sensitivity Analysis · p. 9
E. Paired Statistical Comparison
- Against CNTSeg, PHM-Net significantly improved Dice and Jac, with median gains of 0.40 and 0.51 percentage points, while ASD and AHD differences were not significant.The adjusted p-values for ASD and AHD were 0.8555 for both.Source: E. Paired Statistical Comparison · p. 9
F. Computational Efficiency
- PHM-Net had 85.06M trainable parameters, 4.57% more than the 81.34M baseline, and required approximately 10 s per subject for inference and volume reconstruction.PHOR was parameter-free and added 0.09G FLOPs.Source: F. Computational Efficiency · p. 9
G. Input Modality Analysis
- With the concatenation-based FECCNet baseline, adding DEC to T1w raised Dice from 71.83% to 73.67% and reduced ASD from 0.587 to 0.567 mm.T1w+FA performed slightly better, but the difference was small; additional modalities produced modest, non-monotonic gains, and T1w+DEC was adopted as a compact competitive configuration.Source: G. Input Modality Analysis · p. 9
V. CONCLUSION
- CNsEMD contains 202 subjects acquired at 3T, 5T, and 7T, and the conclusion reports that PHM-Net achieved the best average performance among compared methods under heterogeneous imaging conditions.The work presents CNsEMD as a reproducible benchmark and PHM-Net as a geometry-consistent solution for multimodal CN parcellation.Source: §V · pp. 9–10