Source-linked AI summary
Channel Splitting Network for Single MR Image Super-Resolution
Xiaole Zhao, Yulun Zhang, Tao Zhang, Xueming Zou
TL;DR
MR super-resolution must balance the performance potential of deep models against the difficulty of training them with limited high-quality data. The paper introduces CSN, which splits hierarchical features into residual and dense branches and integrates them through merge-and-run mapping. Across MR settings, CSN is reported to achieve superior performance while addressing this training dilemma to some extent.
Problem
Deep MR super-resolution models are difficult to train effectively because high-quality training samples are scarce, while shallow models often provide unsatisfactory performance.
Method
CSN splits hierarchical features into residual and dense branches with different information propagation and uses merge-and-run mapping to integrate them.
Results
CSN achieves the best SR performance in both C1 and C96 cases under bicubic and k-space truncation degradations.
Takeaways & Limitations
Channel splitting helps increase network depth and diversify hierarchical-feature processing, improving the tradeoff between model performance and training difficulty to some extent.
Abstract
from arXiv · showhide
High resolution magnetic resonance (MR) imaging is desirable in many clinical applications due to its contribution to more accurate subsequent analyses and early clinical diagnoses. Single image super resolution (SISR) is an effective and cost efficient alternative technique to improve the spatial resolution of MR images. In the past few years, SISR methods based on deep learning techniques, especially convolutional neural networks (CNNs), have achieved state-of-the-art performance on natural images. However, the information is gradually weakened and training becomes increasingly difficult as the network deepens. The problem is more serious for medical images because lacking high quality and effective training samples makes deep models prone to underfitting or overfitting. Nevertheless, many current models treat the hierarchical features on different channels equivalently, which is not helpful for the models to deal with the hierarchical features discriminatively and targetedly. To this end, we present a novel channel splitting network (CSN) to ease the representational burden of deep models. The proposed CSN model divides the hierarchical features into two branches, i.e., residual branch and dense branch, with different information transmissions. The residual branch is able to promote feature reuse, while the dense branch is beneficial to the exploration of new features. Besides, we also adopt the merge-and-run mapping to facilitate information integration between different branches. Extensive experiments on various MR images, including proton density (PD), T1 and T2 images, show that the proposed CSN model achieves superior performance over other state-of-the-art SISR methods.
I. INTRODUCTION
High-resolution MR images support clinical analysis but are constrained by acquisition limitations and the difficulty of training deep super-resolution models with scarce high-quality samples. CSN addresses this by splitting hierarchical features into differently structured branches and integrating them with merge-and-run mapping.
- High-resolution MR imaging can provide structural and texture details that support subsequent analysis and early diagnosis.
- Hardware, imaging time, signal-to-noise ratio, and body motion constrain MR acquisition, while higher spatial resolution can reduce SNR or increase imaging time.
- Deep MR super-resolution faces a tradeoff: shallow models are easier to train but often underperform, whereas deeper models are harder to train with MR data.
- CSN splits hierarchical feature maps into residual and dense branches to reduce representational redundancy and support feature reuse and new-feature exploration.
- Merge-and-run mapping integrates information between the branches, while global feature fusion and stabilization techniques support the overall model.
II. RELATED WORK
Deep-learning super-resolution methods have expanded through deeper, recursive, residual, dense, and multi-stream architectures, but their practicality remains constrained by representation and computational tradeoffs. MR super-resolution also requires degradation models that reflect acquisition processes and avoids the calibration burden of multi-frame inputs.
- Super-Resolution with Deep Learning: Deep-learning super-resolution methods developed through increased depth, recursive learning, residual learning, and dense feature reuse.
- Super-Resolution with Deep Learning: Lightweight super-resolution methods improve time-efficiency tradeoffs but remain limited in representational capacity.
- MR Image Super-Resolution: K-space truncation models MR acquisition by reducing phase- and slice-encoding lines, producing degradation patterns distinct from bicubic image-domain shrinking.
- MR Image Super-Resolution: Multi-frame MR super-resolution requires calibration and fusion of multiple degraded low-resolution images, creating a challenging reconstruction step.
C. Multi-Stream Networks
The CSN uses channel splitting to build its multi-stream structure, unlike prior methods that reuse preceding features. Its architecture extracts shallow features, processes them through stacked channel splitting blocks, and then reconstructs the image.
- CSN forms multiple streams by splitting preceding features into different branches, unlike prior multi-stream methods that reuse those features.
- The network comprises feature extraction, nonlinear mapping, and image recovery, with the nonlinear mapping network containing stacked channel splitting blocks.
- The feature extraction network uses two 3×3 convolutional layers with a 1×1 convolutional layer between them to produce shallow features.
- The nonlinear mapping network passes each channel splitting block’s output directly to the next block, forming a continuous-memory mechanism for information propagation.
3) Image Reconstruction:
Image reconstruction fuses local features from all channel splitting blocks, applies global residual learning, and restores the HR image through pixel shuffle, convolution, and external residual learning.
- Local features from all channel splitting blocks are concatenated and fused using a 1×1 convolution followed by a 3×3 convolution.
- Global residual learning adds a global skip connection to stabilize training and produce fused features for HR image recovery.
- HR restoration mainly uses a pixel shuffle layer followed by a 3×3 convolutional layer and external residual learning.
- The pixel shuffle layer is implemented using ESPCNN following the cited implementation approach.
B. Channel Splitting Block
Each channel splitting block divides input features into residual and dense branches, applies different transmission mechanisms, and merges the branches to target hierarchical features while controlling model complexity.
- A channel splitting block divides its input tensor into lower dense and upper residual branches along the channel direction.
- Residual learning supports feature reuse, while dense learning explores new features within the two branches.
- Channel merging recombines the two branch outputs at the end of each block after applying their distinct information-transmission mechanisms.
- The splitting and merging operations target hierarchical features with different properties and help maintain parameter scale while increasing network depth.
- The performance comparison evaluates channel splitting and merge-and-run mapping, different branch structures, and different kernel sizes on V(T1, TD) for SR×2.
2) Feature Reuse and New Feature Exploration:
The CSB combines residual-like and dense-like branches in parallel through merge-and-run mappings, integrating feature reuse and new-feature exploration while supporting information flow and training.
- A CSB assembles two parallel residual-like branches with different structures through merge-and-run mappings.
- The upper branch resembles EDSR, while the lower branch is a simplified DenseNet- or RDB-like structure using one skip dense connection.
- The merge-and-run mapping uses an idempotent transformation that promotes information flow across modules and encourages gradient back-propagation.
- After the stage mappings, the branch outputs are merged and local residual learning is added through a 1×1 convolution.
C. Multilevel Residual Mechanism
The network uses interpolated-input residual learning and multilevel residual mechanisms to stabilize training and improve reconstruction.
- Multilevel Residual Mechanism: The model learns the residual between the HR target and an interpolated LR input because LR and HR images share substantial information.Bicubic interpolation is used, though other interpolation algorithms are also possible.
- Multilevel Residual Mechanism: Combining global shortcut connections and local residual learning gives the network multilevel residual learning.The paper reports that this combination can stabilize training and slightly improve performance.
- Multilevel Residual Mechanism: Multilevel residual learning is especially helpful for single MR image super-resolution because degraded training samples make model training more unstable.
- Multilevel Residual Mechanism: Table I specifies the detailed CSN configuration, with repeated channel-splitting blocks and padded convolutional layers preserving feature-map size.
D. Training Objective and Network Depth
CSN is trained end to end with an L1 reconstruction objective, while its depth depends on the numbers of CSBs and stage mappings; experiments evaluate configurations across MR datasets and degradations.
- Training Objective: The model estimates parameters by minimizing L1 loss between reconstructed HR images and ground-truth HR images.The training set contains paired LR inputs and HR targets.
- Training Objective: L1 loss is selected because it provides better training convergence than L2 loss, particularly when training samples are degraded.
- Network Depth: CSN depth is determined mainly by the number of CSBs and stage mappings, with the pixel-shuffle depth depending on the scaling factor.The pixel-shuffle depth is 1 for SR×2 and SR×3 and 2 for SR×4.
- Experimental Setting: The experiments assess CSN configurations and compare SISR methods on bicubic-downsampling and k-space-truncation degradations using PSNR and SSIM.Datasets are constructed from PD, T1, and T2 MR images in the IXI dataset.
- Scope: The model targets single 2D MR image super-resolution, while extending the generated datasets to 3D algorithms is reserved for future work.
B. Implementation Details
Implementation studies vary CSB and stage-mapping structures to assess channel splitting, merge-and-run mapping, branch design, and model-scale tradeoffs.
- Implementation Details: The implementation study examines stage mappings, multilevel residual learning, global feature fusion, and building-block utilization.
- Stage Mapping: The proposed CSN-R3D3 stage mapping mixes residual and dense branches, with both the number of mappings and CSBs set to 4.
- Stage Mapping: Channel splitting and merge-and-run mapping significantly improve validation performance, while mixed residual-dense branches provide a slight advantage over single-branch alternatives.The comparison uses V(T1, TD) for SR×2, with equal network depth across the compared branch structures.
- Stage Mapping: Larger-kernel alternatives outperform CSN-R3D3 but use about 1.6 times as many parameters, producing a worse performance-versus-scale tradeoff.
- Branch Design: The residual branch favors better performance in the tested stage-mapping structures.
2) External Skip Connection:
External skip connections stabilize training and improve reconstruction, while deeper CSN configurations generally improve performance at the cost of additional parameters; the model is compared with established SISR methods.
- External Skip Connection: ESC-Bicubic performs significantly better than ESC-None, ESC-NN, and ESC-Bilinear on D(PD, BD) for SR×2.
- External Skip Connection: External skip connections stabilize training regardless of the interpolation method and can reduce the possibility of training failure.This is particularly relevant when training examples are degraded.
- Network Capacity: Increasing the number of building blocks improves PSNR on V(T2, TD) for SR×2, but also increases the number of parameters.
- Network Capacity: Models with t building blocks and four stage mappings outperform models with four building blocks and t stage mappings for t = 1, 2, 3, despite using fewer parameters.
- Comparison Methods: The final comparison retrains CSN and several advanced SISR methods across six test datasets covering two degradations and three MR image types.
1) Bicubic Degradation (BD):
Under bicubic downsampling, CSN achieves the best super-resolution performance across C1 and C96 while using fewer parameters and shallower structures than RDN and EDSR. Visual examples show stronger recovery of dark regions, ridges, and contours, while truncation-degradation comparisons also favor CSN overall.
- Bicubic downsampling generates low-resolution images by shrinking high-resolution images with a bicubic kernel.
- CSN gives the best SR performance in both C1 and C96 cases despite fewer parameters and shallower structures than RDN and EDSR.
- In PD, T1, and T2 examples, CSN better restores black areas, gray ridges, and dark ditches than the compared methods.
- Under truncation degradation, CSN reconstructs dark contours that other models cannot clearly recover and performs better overall than EDSR in C96 comparisons.
A. Multiple Branches
The CSN explicitly splits hierarchical features into branches with different information propagation, enabling deeper models while keeping width and parameters nearly unchanged. Its current scope is limited to 2D MR super-resolution with one image type and scaling factor, although broader extensions are proposed.
- Branch structure: The current branch structures are limited to ResNet-like and DenseNet- or RDN-like designs, and mixing them yields only modest performance gains.The authors conjecture that larger structural differences between branches could produce larger performance differences.
- Scope: The work addresses only 2D MR image super-resolution, leaving 3D extension as future work despite the potential value of 3D structural information.The authors note that 3D models can sharply increase parameters as depth grows, while CSN may help extend them.
- Depth and width: Channel splitting enables greater network depth and processing diversity without significantly increasing model width and parameters.This design is intended to ease the trade-off between improving performance and training deep models.
- Information sharing: Experiments are restricted to a single type of 2D MR image and one scaling factor, excluding possible benefits from combining image types and scales.The paper identifies multi-type and multi-scale information as a prospective way to improve performance.
- Branch design: CSN splits hierarchical features into residual and dense branches with different information propagations, then merges branch features through MAR mapping.The residual and dense branches support different feature-processing behaviors, while MAR integrates their hierarchical features.