Source-linked AI summary
Implicit Neural Representation in Medical Imaging: A Comparative Survey
Amirali Molaei, Amirhossein Aminimehr, Armin Tavakoli, Amirhossein Kazerouni, Bobby Azad, Reza Azad, Dorit Merhof
TL;DR
Medical imaging contains challenging tasks and costly annotation requirements, motivating a survey of implicit neural representations in this domain. The paper organizes INR principles, applications, advantages, limitations, and research opportunities across medical imaging. It concludes that INRs show substantial potential while remaining constrained by computational demands and the time required to fit high-dimensional signals.
Problem
Medical imaging involves challenging tasks and expensive, time-consuming expert annotation, creating motivation for methods that can use measurements without external training annotations.
Method
The paper provides a systematic review and taxonomy of INR principles and medical-imaging applications, comparing state-of-the-art approaches and discussing open challenges and future directions.
Results
The survey finds that image reconstruction has attracted more INR research than segmentation, compression, registration, and other tasks, particularly because INRs can enhance resolution and reduce noise.
Takeaways & Limitations
INRs show substantial potential for complex medical-imaging problems across reconstruction, segmentation, registration, neural rendering, and related applications.
Takeaways & Limitations
Fitting a separate neural representation for each signal can require considerable computational resources and time, especially for high-dimensional data such as 3D volumes.
Abstract
from arXiv · showhide
Implicit neural representations (INRs) have gained prominence as a powerful paradigm in scene reconstruction and computer graphics, demonstrating remarkable results. By utilizing neural networks to parameterize data through implicit continuous functions, INRs offer several benefits. Recognizing the potential of INRs beyond these domains, this survey aims to provide a comprehensive overview of INR models in the field of medical imaging. In medical settings, numerous challenging and ill-posed problems exist, making INRs an attractive solution. The survey explores the application of INRs in various medical imaging tasks, such as image reconstruction, segmentation, registration, novel view synthesis, and compression. It discusses the advantages and limitations of INRs, highlighting their resolution-agnostic nature, memory efficiency, ability to avoid locality biases, and differentiability, enabling adaptation to different tasks. Furthermore, the survey addresses the challenges and considerations specific to medical imaging data, such as data availability, computational complexity, and dynamic clinical scene analysis. It also identifies future research directions and opportunities, including integration with multi-modal imaging, real-time and interactive systems, and domain adaptation for clinical decision support. To facilitate further exploration and implementation of INRs in medical image analysis, we have provided a compilation of cited studies along with their available open-source implementations on \href{https://github.com/mindflow-institue/Awesome-Implicit-Neural-Representations-in-Medical-imaging}. Finally, we aim to consistently incorporate the most recent and relevant papers regularly.
1. Introduction
Implicit representations encode signals as continuous coordinate-to-value functions, offering memory-efficient and resolution-agnostic alternatives to explicit grids. This survey reviews their growing use in medical imaging, organizes applications and architectures, and identifies associated challenges and research directions.
- Implicit neural representations: Implicit representations map input coordinates to signal values through a generator function rather than discretizing the input space.This continuous formulation contrasts with explicit representations such as point clouds, voxel grids, and meshes.
- Implicit neural representations: MLPs parameterize images, shapes, and other signals as implicit functions, supporting applications including image generation, super-resolution, and 3D reconstruction.Coordinates serve as inputs to an MLP trained to represent the signal of interest.
- Advantages: Continuous-domain operation makes INRs resolution-agnostic and allows values to be generated between pixel- or voxel-wise grid locations.Their representation can therefore support vision tasks without being tied to a fixed discrete resolution.
- Medical imaging applications: INRs attract medical-imaging research because they can reduce dependence on labeled data while addressing resolution enhancement, missing-information synthesis, and inverse problems.Applications include reconstructing CT and MRI data from projection and frequency domains, including settings involving reduced radiation exposure.
- Survey scope: The survey systematically reviews INR applications, compares state-of-the-art approaches, develops a taxonomy, and discusses limitations, open issues, and future directions.The paper also prioritizes pioneering and highly ranked studies in its review process.
2. Background
INRs represent signals continuously with neural networks, while Fourier features, sinusoidal activations, and multi-head architectures address their tendency to miss high-frequency or local details. NeRF extends this framework to novel-view synthesis by mapping spatial coordinates and viewing directions to density and color for volume rendering.
- 2. Background: INRs use neural networks to map spatial coordinates continuously to signal values instead of relying on grids, voxels, point clouds, or meshes.The implicit function maps coordinates in M-dimensional space to corresponding N-dimensional values such as occupancy or color.
- 2. Background: Conventional coordinate-based MLPs tend to learn low-frequency functions, limiting their ability to represent complex signals and fine details.This limitation is associated with spectral bias in piece-wise linear networks.
- 2.1. Input: Sinusoidal mappings and Fourier features map Cartesian coordinates into higher-dimensional spaces to improve high-frequency signal representation.The encoding scale is selected through a hyperparameter sweep for each task and dataset.
- 2.2. Activation Function: SIRENs replace conventional activations with sine functions, enabling representation of high-frequency details and efficient parameterization of higher-order derivatives.The sine derivative is a shifted sine, or cosine, which supports derivatives such as image Laplacians or Helmholtz equations.
- 2.3. Network Architecture: Multi-head networks separate global and local learning by using a shared main body and multiple output heads that reconstruct different signal parts.For images, the heads learn local features of separate regions.
- 2.4. NeRF: NeRF combines implicit scene representation with volume rendering, using spatial coordinates and viewing directions to predict density and color for novel-view synthesis.Its two-stage design first predicts density and features from 3D coordinates, then predicts color before tracing camera rays through the target image.
3. Clinical Importance
INRs address major medical-imaging constraints involving expert annotation, sparse measurements, memory, and difficult clinical scenes. Their applications include reconstruction, super-resolution, robotic surgery, and expert-validated enhancement of anatomical structures.
- 3. Clinical Importance: Medical-image annotation is expensive, time-consuming, privacy-constrained, and dependent on clinicians, creating a major bottleneck for automated imaging.INRs can support several applications without external training annotations.
- 3. Clinical Importance: INRs support reconstruction of CT and MRI scans directly from sensor-domain observations and can incorporate earlier scans to track tissue progression.These applications address inverse imaging problems involving recovery of organ structure from measurements.
- 3. Clinical Importance: Reconstructing images from sparse data can reduce radiation dose in CT and accelerate MRI scans, while SISR restores high-resolution images from single inputs.The passage identifies sparse-data reconstruction as practically important across medical imaging applications.
- 3. Clinical Importance: INRs assist robotic surgery by improving interpretation of intraoperative images, tissue segmentation, anatomical localization, and surgical-tool manipulation.The described systems provide real-time feedback and guidance to surgeons.
- 3. Clinical Importance: An INR-enhanced QSM atlas improved the spatial resolution of a Parkinson’s-disease PPN atlas, reducing artifacts and blurring and enabling clearer anatomical delineation validated by radiologists.The study compared aligned atlas regions with manual radiologist segmentations.
- 3. Clinical Importance: INRs address limited annotations and scan artifacts across diverse imaging challenges, with expert validation supporting assessment of their practical usefulness.The survey presents INRs as adaptable tools whose clinical use is expected to grow.
4. Taxonomy
The survey organizes medical-imaging INR applications across reconstruction, segmentation, registration, compression, and neural rendering, emphasizing task-specific formulations and representative methods. These methods address sparse or incomplete measurements, complex anatomical structures, deformation, and biomedical-data compression.
- Taxonomy: The taxonomy covers reconstruction, segmentation, registration, compression, and neural rendering as major medical-imaging INR application areas.The figure lists these application sub-fields and associates methods with ascending reference numbers.
- Reconstruction: Medical image reconstruction is framed as an inverse problem that maps noisy or undersampled measurements to a complete image.The survey contrasts this with conventional reconstruction methods that require large datasets and may generalize poorly across modalities or anatomy.
- Reconstruction: NeRP reconstructs sparsely sampled CT or MRI data without training data by embedding a prior image in network weights, adapting to measurements, and querying coordinates.The framework uses a three-stage process: prior-image embedding, target-measurement optimization, and image inference.
- Segmentation: INR-based segmentation combines CNNs with implicit representations to enhance retinal-image resolution and preserve detailed vessel topology using an improved clDice loss.On the DRIVE and CHASE datasets, BS-ISR was reported to outperform non-INR methods in segmentation accuracy, detail detection, and robustness.
- Registration: IDIR models deformable registration with an INR-based SIREN MLP that directly represents the transformation function mapping coordinates between images.Its periodic activation supports higher-order derivatives and advanced regularization; on 4D chest CT, it surpassed deep-learning methods without folding or training data.
- Compression and neural rendering: INR methods also target biomedical compression and dynamic scene modeling, with TINC outperforming conventional compression techniques but retaining slower compression speed.Surgical neural rendering represents deformable scenes through canonical radiance and time-dependent displacement fields implemented with MLPs.
5. Comparative Overview
The survey compares INR methods across reconstruction, segmentation, neural rendering, compression, and registration, with reconstruction receiving the greatest attention. It highlights task-dependent input parameters, sparse-view reconstruction trade-offs, and network choices shaped by representation objectives.
- Comparative Overview: Image reconstruction attracts more interest because INRs can enhance resolution and reduce noise in uncertain medical imaging.
- Comparative Overview: Table 1 compares INR methods across reconstruction, segmentation, neural rendering, compression, and registration.The comparison is organized by medical imaging task.
- Defining Parameters: INR inputs are task-dependent and may use sensor-response parameters, viewing angles, detector locations, or cardinal-direction distances instead of Cartesian coordinates.CoiL models measurement fields using sensor-response parameters, while NeRD uses positional distances in cardinal directions.
- Sparse View CT Reconstruction: Sparse-view CT reconstruction trades reduced radiation exposure for potential losses in image quality and accuracy, with limited projections producing blurry outputs.NeRP and CoiL leverage prior information or geometric relationships to address sparse reconstruction.
- Network Type: Network selection depends on the task: ReLU MLPs with Fourier features model 3D scenes, whereas periodic activations support higher-order registration regularizers.The cited regularizers include Jacobian, hyperelastic, and bending-energy penalties.
6. Future Work and Open Challenges
The survey identifies computational cost, training time, complex-signal scaling, and high-frequency video modeling as obstacles to practical INR deployment. It points to meta-learning and multi-scale representations as approaches that may improve efficiency, while dynamic semantic modeling remains challenging.
- Computational Complexity and Training Time: Fitting a separate INR for each signal requires substantial memory and computation, and high-dimensional 3D volumes can take considerable time to train.These costs challenge applications requiring immediate responses.
- Computational Complexity and Training Time: Meta-learning and multi-scale representations can accelerate training and improve memory utilization for anatomical and biological structures.
- Scaling to More Complex Signals: Representing higher-resolution signals or complex 3D shapes with fine detail remains challenging because the mapping can fail to encode high-frequency information.INRs may therefore produce suppressed representation quality relative to discrete representations.
- Dynamic Clinical Scenes: Although INR compression supports parallel frame decoding and has drawn attention in robotic-assisted surgery, semantic relationships across high-frequency video frames remain difficult to model.
7. Conclusion
The survey reviews how neural networks and implicit continuous functions address complex medical imaging problems across multiple tasks. It also identifies INR benefits, open challenges, and future research directions for the field.
- Conclusion: The survey provides a comprehensive overview of INRs in medical imaging and their applications across diverse medical imaging tasks.
- Conclusion: It emphasizes the potential of neural networks and implicit continuous functions for addressing complex problems in medical settings.
- Conclusion: The survey identifies open challenges and future research areas for researchers studying INRs in medical imaging.