PaperlayerView exact revision on arXiv

Source-linked AI summary

MedGSSR: Generalizable Medical Image Super-Resolution 3D Reconstruction via Hierarchical Feed-forward Gaussian Splatting

Chengkai Wang, Luoyu Hong, Yiting Zhao, Jiamin Wang, Xiang Feng, Feiwei Qin, Zhenzhong Kuang, Xuefei Yin, Ali Bashashati, Yanming Zhu

arXiv:2609.06874v1eess.IVcs.CEcs.CVq-bio.QM

TL;DR

Medical 3D super-resolution seeks to recover high-resolution volumes when scanner hardware, scan time, and CT radiation dose constrain acquisition, while existing approaches face fidelity and generalization bottlenecks. MedGSSR predicts a continuous 3D Gaussian field feed-forward, and the paper reports improved MRI/CT reconstruction and performance on unseen datasets.

  • Problem

    Medical 3D super-resolution aims to recover missing spatial information when hardware, scan time, and CT radiation dose constrain high-resolution acquisition; existing methods face fidelity and generalization challenges.

    Source: §1 · p. 2

  • Method

    MedGSSR uses feed-forward prediction of a continuous 3D Gaussian field, with hierarchical structural and textural branches and sub-voxel decomposition.

    Source: §1 · p. 3, §4.3 · p. 7

  • Results

    MedGSSR outperforms state-of-the-art methods on MRI and CT benchmarks and improves PSNR over ArSSR by 3.97 dB at 4× on held-out HCP.

    Source: §1 · p. 3, §5.3 · p. 12

  • Takeaways & Limitations

    The framework reconstructs unseen subjects without test-time optimization and operates in the 3D domain without relying on 2D proxy projections.

    Source: §1 · p. 3, §6 · p. 15

Abstract

from arXiv · show

High-resolution volumetric medical imaging is critical for clinical diagnosis, yet acquisition is often limited by scanner hardware, scan time, and for CT, radiation dose. Medical 3D Super-Resolution (Med3DSR) offers a computational alternative, but existing methods commonly rely on per-subject optimization, pretrained priors, or coordinate-based implicit representations, which compromise anatomical fidelity and limit efficiency. To address these limitations, we present MedGSSR, a fully end-to-end feed-forward framework that represents volumes as an explicit 3D Gaussian field for Med3DSR. Unlike coordinate-based implicit functions, our explicit 3D Gaussian representation naturally enhances signal continuity and local high-frequency fidelity. Specifically, MedGSSR explicitly decouples the reconstruction process into coarse-grained structural preservation and fine-grained textural refinement through the proposed Pyramid Anatomical Encoder and a Hierarchical Gaussian Projector. To support arbitrary-scale super-resolution, we introduce sub-voxel Gaussian decomposition and a Differentiable Gaussian Voxelizer that directly queries the continuous 3D intensity field, reducing discretization artifacts. Extensive experiments on MRI and CT benchmarks demonstrate that MedGSSR significantly outperforms state-of-the-art methods. Notably, our framework exhibits robust generalizability across unseen datasets without requiring per-subject optimization, enabling fast inference and high-fidelity volumetric super-resolution in practical clinical settings. Our project webpage, including code, is at https://william2ai.github.io/medgssr

1 Introduction

MedGSSR reframes medical 3D super-resolution as feed-forward prediction of a continuous volumetric Gaussian field. Its hierarchical design separates anatomical structure from texture, and the paper reports improved reconstruction and cross-dataset generalization.

1 Introduction

  • High-resolution MRI and CT acquisition is constrained by scarce high-field scanners, long scan times, motion-artifact risk, and CT radiation dose.Med3DSR aims to computationally recover spatial information without changing imaging hardware.Source: §1 · p. 2
  • Coordinate-based INRs can oversmooth fine anatomy, while per-subject optimization is costly and 2D proxy supervision can introduce inter-slice inconsistencies.The paper identifies limited isotropic 3D medical data as a barrier to learning 3D-consistent priors.Source: §1 · pp. 2–3
  • MedGSSR uses local-support Gaussian primitives to model high-frequency structures and continuous medical-image densities without test-time optimization.The authors describe this representation as physically consistent for measured densities such as CT Hounsfield units and MRI proton density.Source: §1 · p. 3
  • The method combines multi-scale volumetric features with separate coarse structural and fine textural branches, sub-voxel Gaussian decomposition, and continuous-field voxelization.The resulting field can be queried at desired spatial coordinates for arbitrary-resolution rendering.Source: §1 · p. 3
  • MedGSSR outperforms state-of-the-art approaches in high-frequency reconstruction fidelity and robust cross-dataset generalization.The proposed mapping predicts an explicit 3D Gaussian field from sparse low-resolution voxel grids.Source: §1 · p. 3

2 Related Work

Prior 3D reconstruction and medical super-resolution methods trade off optimization cost, spatial consistency, fidelity, and scale flexibility. MedGSSR adapts feed-forward Gaussian prediction to continuous medical volumes.

2.1 3D Reconstruction

  • 3DGS uses explicit point-based rasterization for efficient rendering, while Mip-Splatting addresses aliasing artifacts in that representation.Earlier reconstruction work also advanced implicit methods with multi-scale representations, hash-grid encodings, and tensorial decomposition.Source: §2.1 · p. 4
  • Natural-scene feed-forward 3DGS predicts from sparse images, whereas MedGSSR adapts the paradigm to medical volumes and 3D-consistent structural priors.Earlier 3DSR approaches relied on per-subject optimization and 2D pseudo-HR labels, which lack 3D spatial awareness.Source: §2.1 · p. 4

2.2 Medical Image Super-Resolution

  • Early 3D CNN methods restored local textures but were computationally demanding and restricted to fixed integer scales,,.Arbitrary-scale INRs,, and self-supervised frameworks followed, but the latter do not leverage large-scale anatomical priors.Source: §2.2 · p. 4
  • Projection-based MedNeRF and NAB-GS [20] exploit 2D generative priors, but their 3D-to-2D-to-3D pipelines introduce modality gaps and geometric ambiguity.MedGSSR instead operates in the continuous 3D domain, avoiding those projection issues as well as the spectral bias of implicit approaches.Source: §2.2 · pp. 4–5

3 Preliminaries

The preliminaries describe 3D Gaussian primitives and the shift from per-subject fitting to feed-forward prediction. MedGSSR maps low-resolution volumes to continuous Gaussian fields for volumetric reconstruction.

3.1 3D Gaussian Splatting (3DGS)

  • Standard 3DGS represents a scene with anisotropic Gaussian primitives parameterized by position, covariance, opacity, and appearance features.The parameters are optimized by gradient descent against reconstruction error between rendered projections and ground-truth images.Source: §3.1 · p. 5

3.2 Feed-Forward 3D Gaussian Prediction

  • MedGSSR formulates medical super-resolution as direct prediction of Gaussian attributes from an encoded low-resolution voxel grid.The mapping predicts a dense, continuous field of primitives directly from 3D scans.Source: §3.2 · p. 5
  • The overview routes coarse and 2×-upsampled fine features into structural and textural Gaussian streams, then aggregates them into a continuous field for HR rendering.This feed-forward formulation replaces standard per-subject 3DGS optimization for volumetric medical data.Source: §3.2 · p. 5, Figure 2 · p. 6

4 Methodology

MedGSSR maps a low-resolution medical volume to a continuous explicit field of 3D Gaussian primitives. Its encoder and projector separate global anatomical structure from local texture, while the voxelizer queries the field at arbitrary target resolutions for end-to-end reconstruction.

4.1 Overview

  • MedGSSR learns a direct mapping from a low-resolution volume to a continuous explicit field of 3D Gaussian primitives.Source: §4.1 · p. 6

4.2 Pyramid Anatomical Encoder

  • The Pyramid Anatomical Encoder uses a 3D U-Net dual-stream design to extract coarse global anatomy and finer local appearance features.The fine stream concatenates the low-resolution input with decoder features before convolution and 2× PixelShuffle upsampling.Source: §4.2 · pp. 6–7

4.3 Hierarchical Gaussian Projector (HGP)

  • The Hierarchical Gaussian Projector uses coarse and fine heads to represent organ-scale structure and refine local textures, with sub-voxel decomposition adding primitives within each voxel.Structural Gaussians use larger scales and higher amplitudes, while textural Gaussians use smaller scales and lower amplitudes; multiple learnable-offset primitives capture sub-voxel transitions.Source: §4.3 · pp. 7–8

4.4 Differentiable Gaussian Voxelizer

  • The Differentiable Gaussian Voxelizer queries the continuous intensity field on a target grid, supporting arbitrary upsampling scales with one trained model.It renders the volume by evaluating the field at grid coordinates sized for the requested scale.Source: §4.4 · p. 8

4.5 Optimization Objective

  • The model is trained end-to-end by minimizing voxel-wise L1 error between its rendered prediction and the ground-truth high-resolution volume.The loss is applied to the prediction sampled at the target high-resolution grid.Source: §4.5 · p. 9

5 Experiments

Experiments evaluate MedGSSR on MRI and CT, both within-domain and across unseen datasets. The results report high reconstruction fidelity, cross-dataset generalization, downstream segmentation performance, and ablations of the model’s design.

5.1 Experimental Setup

  • The evaluation uses MSD MRI and MELA CT for intra-domain experiments, and HCP MRI and UHRCT CT as held-out generalization datasets.Comparisons include interpolation, self-supervised, neural implicit, and feed-forward 3DGS methods; reconstruction metrics are PSNR, SSIM, and LPIPS.Source: §5.1 · p. 9

5.2 Main Results

5.3 Generalization Results

  • On unseen HCP at 4×, MedGSSR improves PSNR by 3.97 dB over ArSSR; on UHRCT at 8×, it gains 5.86 dB over ArSSR and 4.74 dB over CuNeRF.Models are evaluated on held-out targets without fine-tuning; the cross-domain tables cover HCP and UHRCT.Source: Table 3 · p. 12, Table 4 · p. 12, §5.3 · p. 12
  • On 4× HCP MRI, MedGSSR achieves the highest average nnUNet Dice score, 0.8178, and its reconstructed boundaries align with the original ground truth.CuNeRF scores 0.3618, while MedGSSR’s average Dice score is reported as a 3.4% relative increase over ArSSR.Source: §5.3 · pp. 12–13, Table 5 · p. 13

5.4 Ablation Study

  • Adding the fine branch to the coarse-only model raises PSNR from 35.24 to 36.60 dB and SSIM from 0.9802 to 0.9870, while lowering LPIPS from 0.0789 to 0.0363.The coarse branch captures global anatomy; the fine branch models high-frequency residuals and restores fine-scale boundaries.Source: §5.4 · p. 13, Figure 6 · p. 14
  • The sub-voxel count m = 4 gives the best reported PSNR, 36.60 dB; m = 8 yields 36.52 dB, with marginal saturation and slightly worse metrics.The authors select m = 4 to balance reconstruction quality and computational cost.Source: §5.4 · pp. 13–14, Table 6 · p. 15
  • Adding FFT loss improves LPIPS from 0.1253 to 0.1078 but slightly decreases PSNR and SSIM; reducing Gaussian truncation to 1σ degrades all metrics.The reported FFT-loss change indicates a trade-off between perceptual sharpness and voxel-wise fidelity; the authors also report robustness with 75% of training data, with performance dropping more noticeably at 50%.Source: §5.4 · pp. 14–15, Table 7 · p. 15

6 Conclusion

The conclusion presents MedGSSR as feed-forward volumetric super-resolution using explicit 3D Gaussian Splatting. It highlights high-fidelity MRI and CT results, cross-dataset generalization, and downstream clinical-task benefits.

6 Conclusion

  • MedGSSR formulates medical 3D super-resolution as generalized volumetric reconstruction using explicit 3D Gaussian Splatting as continuous basis functions.The framework removes isolated per-volume optimization and alleviates over-smoothing commonly observed in implicit neural representations.Source: §6 · p. 15
  • A Hierarchical Gaussian Projector with sub-voxel decomposition separates global structural preservation from fine-scale detail refinement.A Differentiable Gaussian Voxelizer enables end-to-end 3D-domain supervision without relying on 2D proxy projections.Source: §6 · p. 15
  • Experiments across MRI and CT benchmarks report high fidelity and robust cross-dataset generalization under domain shift, with structural accuracy benefiting downstream clinical tasks.The conclusion describes MedGSSR as a potential efficient and reliable foundation for volumetric medical image enhancement.Source: §6 · p. 15

A Additional Visual Comparisons for Intra-Domain 3D Super-Resolution

Additional intra-dataset visual comparisons cover MRI and CT at 2×, 3×, and 4× across anatomical planes. They report clearer boundaries and local details with MedGSSR than with competing methods.

A Additional Visual Comparisons for Intra-Domain 3D Super-Resolution

  • On MSD MRI, MedGSSR restores sharper cortical folds and tissue boundaries across axial, coronal, and sagittal views, including at 4×.Interpolation becomes increasingly blurry with scale, while CuNeRF and ArSSR show over-smoothing and lose fine folding patterns.Source: A Additional Visual Comparisons for… · p. 19
  • On MELA CT, MedGSSR better preserves anatomical boundaries and local details across scales and planes, with fewer artifacts and reconstructions closer to ground truth.The comparisons include zoomed regions; competing interpolation, CuNeRF, and ArSSR results show blurred boundaries or difficulty recovering fine structures.Source: A Additional Visual Comparisons for… · p. 19
  • Across the intra-dataset MRI and CT visual comparisons, MedGSSR preserves global anatomy and local high-frequency detail.The examples show axial, coronal, and sagittal views with enlarged regions at 2×, 3×, and 4× scales.Source: A Additional Visual Comparisons for… · p. 19

B Additional Visual Comparisons for Cross-Domain 3D Super-Resolution Generalization

Additional cross-dataset visual comparisons examine unseen HCP MRI and UHRCT CT. The passages report that MedGSSR retains clearer boundaries and local structures under dataset shift and at larger upsampling factors.

B Additional Visual Comparisons for Cross-Domain 3D Super-Resolution Generalization

  • The additional comparisons show MedGSSR’s reported visual robustness on unseen MRI and CT datasets, including at large upsampling factors.The accompanying figures use multiple anatomical planes for HCP MRI and the axial plane for UHRCT CT.Source: B Additional Visual Comparisons for… · pp. 19, 22
  • On unseen HCP MRI, models trained on MSD show that MedGSSR preserves clearer tissue boundaries, coherent cortical patterns, and local details across three anatomical planes.The comparisons cover 2×, 3×, and 4× scales; CuNeRF and ArSSR show degradation in fine structures under dataset shift.Source: B Additional Visual Comparisons for… · pp. 19, 22
  • On unseen UHRCT CT, MedGSSR retains clearer anatomical boundaries and local structures at 4×, 6×, and 8× despite increasing resolution gaps.Models are trained only on MELA, and the comparisons visualize the axial plane under the benchmark’s native acquisition setting.Source: B Additional Visual Comparisons for… · p. 22

C Extended Baseline and Efficiency Analysis on Cross-Domain HCP MRI

On unseen HCP MRI, MedGSSR combines the strongest reported PSNR and SSIM with low computational cost. Additional visual comparisons test cross-dataset reconstruction on unseen HCP and UHRCT data across multiple upsampling scales.

C Extended Baseline and Efficiency Analysis on Cross-Domain HCP MRI

  • Trained on MSD and evaluated on unseen HCP MRI at 2×, 3×, and 4×, MedGSSR better preserves tissue boundaries, cortical structures, and local details under dataset shift [26].The comparisons span axial, coronal, and sagittal views.Source: Figure 9 · p. 23
  • Trained on MELA and evaluated on unseen UHRCT at 4×, 6×, and 8×, MedGSSR retains clearer anatomical boundaries and finer local structures as the resolution gap increases [7].The comparisons use axial views with zoomed-in regions.Source: Figure 10 · p. 24
  • MedGSSR achieves the best PSNR and SSIM, fewest FLOPs, and shortest inference time on cross-domain 4× HCP MRI, while ArSSR has the lowest LPIPS and transformer baselines incur higher cost.MTVNet and SuperFormer provide competitive reconstruction quality but require substantially higher FLOPs and inference time.Source: C Extended Baseline and Efficiency… · p. 22, Table 8 · p. 24

D Additional Visualization of Downstream Segmentation on Super-Resolved MRI Volumes

Additional cross-domain HCP MRI visualizations examine brain tissue segmentation from 4× and 8× super-resolved volumes across anatomical views. MedGSSR's masks remain more faithful to ground truth, including in the more challenging 8× setting.

D Additional Visualization of Downstream Segmentation on Super-Resolved MRI Volumes

  • The additional visualizations extend the main paper's 4× representative result with more comprehensive views of downstream segmentation on unseen HCP data.The same segmentation setting as in the main paper is used.Source: D Additional Visualization of… · p. 24
  • At 4×, MedGSSR's HCP segmentation masks have clearer boundaries and better preserve fine anatomical regions, while competing methods show blurring, boundary leakage, or structural distortion.The visual comparisons span axial, coronal, and sagittal views.Source: D Additional Visualization of… · pp. 24–25, Figure 11 · p. 25
  • At 8×, baseline segmentation masks degrade with boundary confusion, region bleeding, and reduced structural consistency, while MedGSSR remains closer to ground truth across all three views.This is presented as a stress test beyond the main experiments.Source: D Additional Visualization of… · p. 25

E Additional CT Ablation Study

Additional intra-domain 4× CT ablations on MELA show that sub-voxel count and Gaussian support affect reconstruction quality, while training with reduced data retains performance close to the full-data model.

E Additional CT Ablation Study

Related papers

Loading 2609.06874v1…