Source-linked AI summary
UnCapsTSR: An Unsupervised Transformer-based Image Super-Resolution Approach for Capsule Endoscopy Images
Anjali Sarvaiya, Shubh Kawa, Lalit Agrawal, Jagrit Joshi, Kishor Upla, Kiran Raja
TL;DR
WCE’s hardware and transmission constraints produce coarse-resolution images, while supervised super-resolution requires unavailable true LR-HR pairs or relies on synthetic degradations that differ from real observations. UnCapsTSR addresses this with an unsupervised transformer-based GAN using dual discriminators, BTV loss, and texture-distribution priors for WCE super-resolution.
Problem
WCE hardware limitations produce coarser-resolution images, while true LR-HR pairs are impractical and synthetic degradations can differ from real observations.
Method
UnCapsTSR uses an unsupervised transformer-based GAN with dual discriminators, BTV loss, and texture-distribution priors without explicit degradation estimation or true LR-HR pairs.
Results
UnCapsTSR is presented as a WCE super-resolution framework that improves spatial continuity in final super-resolved images while avoiding genuine LR-HR paired datasets.
Takeaways & Limitations
The framework supports WCE super-resolution when genuine LR-HR training pairs and explicit real-world degradation estimation are unavailable.
Takeaways & Limitations
Synthetic LR-HR training can suffer distribution shift from naturally observed LR data, limiting generalizability to real-world observations.
Abstract
from arXiv · showhide
Wireless Capsule Endoscopy (WCE) captures and streams video while passing through a patient's Gastrointestinal (GI) tract and is used to examine its irregularities. Although advantageous over conventional endoscopy, WCE suffers from limitations related to capsule size and wireless transmission, resulting in images with coarser resolution. This work presents UnCapsTSR, an unsupervised transformer-based Generative Adversarial Network (GAN) framework for improving the spatial resolution of Low-Resolution (LR) WCE images. The proposed method accomplishes SR without explicit degradation estimation of real-world LR data and eliminates the need for true LR-HR pairs. UnCapsTSR employs a Bilateral Total Variation (BTV) loss to ensure spatial continuity in SR images. A newly curated dataset from the Kvasir Capsule dataset is also presented for training WCE SR models. Generalizability is validated on KID and GIANA datasets that are not used during training. A new non-reference metric, Endoscopy Quality Metric (EndoQM), is introduced for quantitative evaluation of domain-specific WCE data. Experiments demonstrate consistent improvement over state-of-the-art unsupervised SR approaches using NIQE, BRISQUE, PIQE, and EndoQM. Statistical evaluation shows 40 to 80 percent improvement in EndoQM from LR to SR across the evaluated datasets.
1. Introduction
Wireless Capsule Endoscopy produces low-resolution imagery because of capsule hardware constraints, while paired LR-HR data and accurate degradation models are difficult to obtain. UnCapsTSR addresses these challenges with an unsupervised transformer-GAN framework, a BTV loss, a curated dataset, and domain-specific evaluation.
- Motivation: WCE hardware limitations hinder acquisition of high-resolution images for gastrointestinal examination.The capsule’s limited size, battery, field of view, and CMOS camera constrain image quality.
- Motivation: Paired LR-HR endoscopy images are difficult to acquire, and synthetic bicubic degradation may not represent real-world observations.This distribution mismatch can reduce the generalizability of supervised SR models to real WCE data.
- Proposed approach: UnCapsTSR uses an unsupervised transformer-based GAN with a transformer generator and dual discriminators for WCE super-resolution without explicit degradation estimation.The architecture captures global and local dependencies while refining high-frequency details and structural consistency.
- Proposed approach: BTV loss is integrated with adversarial and texture losses to preserve spatial continuity while retaining edge sharpness in SR images.The loss smooths shifted pixel neighborhoods and penalizes inconsistencies without discarding diagnostically relevant detail.
- Data and evaluation: A derivative SR dataset is curated from Kvasir Capsule by removing redundant or low-quality images and cropping unwanted border pixels.The resulting dataset is designed specifically for training and evaluating WCE SR models.
- Data and evaluation: Generalizability is evaluated on unseen KID and GIANA datasets, with improvements reported across BRISQUE, NIQE, PIQE, and EndoQM for real LR samples.EndoQM is introduced as a reference-less metric focused on edge details relevant to WCE assessment.
2. Background and Related Works
Single-image super-resolution has progressed from interpolation and CNN methods toward Transformer-based models, but WCE applications remain underexplored because real LR-HR pairs are difficult to obtain and synthetic degradations differ from natural observations.
- Background: Traditional SISR methods struggled with noise, aliasing, blurring, high scale factors, and spectral-information preservation.Learning-based methods addressed some of these limitations using external data.
- Limitations of supervised SR: CNN and GAN SR models commonly train on synthetically degraded LR images paired with HR images, risking a distribution shift from real LR observations.Known degradations such as bicubic downsampling may not represent naturally captured WCE data.
- Unsupervised SR approaches: Unsupervised SR methods use degradation modeling or unpaired LR-HR data, but medical-imaging applications remain limited, with MDASR identified as the only unsupervised WCE-focused work.Existing medical studies also target modalities such as CT and MRI, while WCE has fewer established benchmarks and datasets.
- Limitations of supervised SR: True LR-HR pairs are often impossible to acquire in medical imaging, motivating alternative strategies for improving spatial resolution.The constraint is especially relevant where repeated investigations for the same condition are impractical.
- Limitations of supervised SR: Synthetic LR generation with known degradation can yield poor real-world learning because synthetically degraded and naturally observed images have significantly different distributions.This distribution difference is noted for supervised deep SR models trained without genuine LR-HR pairs.
- Transformer-based SR: Transformers capture global dependencies through self-attention, yet they have not been extensively explored for medical-image super-resolution.CNNs primarily model local spatial relationships, whereas Transformer methods can model broader context.
3. Proposed Framework: UnCapsTSR
UnCapsTSR is an unsupervised transformer-based GAN framework for WCE super-resolution using unpaired LR and HR images. Its architecture and losses combine global-local feature modeling, dual-discriminator refinement, texture consistency, and spatial continuity.
- Proposed Framework: UnCapsTSR: UnCapsTSR performs WCE super-resolution with transformer-based generative adversarial learning using one generator and two discriminators.The framework is designed for unsupervised training without explicit degradation estimation or true LR-HR pairs.
- Generator Architecture: The transformer generator replaces a CNN backbone with recursive-generalization self-attention modules to capture long-range dependencies in WCE images.The generator architecture is organized into low-level feature extraction, high-level feature extraction, and image reconstruction modules.
- Loss Functions: UnCapsTSR combines color, adversarial, texture, and Bilateral Total Variation losses to optimize reconstruction under unpaired supervision.The BTV loss penalizes inconsistencies among neighboring pixels, while texture loss uses HR image statistics as domain-level priors.
- Validation: The framework is validated across KID, GIANA, and retinal-image benchmarks to assess robustness across acquisition settings and medical modalities.The comparison with DUSGAN emphasizes broader medical validation beyond visible natural imagery.
- Discriminator Framework: The dual discriminators separately refine high-frequency details and structural-perceptual consistency in generated super-resolved images.Discriminator-I focuses on edges and textures, while Discriminator-II evaluates generated SR images against unpaired HR images.
- Generator Architecture: The generator fuses low- and high-frequency features through a residual connection before pixelshuffle-based image reconstruction.This hierarchical design preserves low-level details while enriching high-level representations for SR output generation.
4. Proposed Quality Metric - EndoQM
EndoQM is a domain-specific no-reference metric introduced to evaluate super-resolved endoscopic images. It addresses the mismatch between natural-image assumptions in NIQE and the constrained statistical characteristics of WCE imagery.
- Proposed Quality Metric - EndoQM: EndoQM is a quality metric specifically tailored to evaluate super-resolved endoscopic images.Its design is motivated by limitations of generic natural-image quality assessment for WCE data.
- Motivation: NIQE models natural-scene statistics, whose textures, illumination patterns, and structural distributions differ from those of WCE images.The passage describes WCE image statistics as narrower and more constrained than natural-image distributions.
- Motivation: EndoQM is intended to provide quantitative quality evaluation that better reflects the domain characteristics of super-resolved endoscopic imagery.The metric is non-reference and specifically designed for medical endoscopy rather than generic natural scenes.
5. Experimental Evaluations
The experiments evaluate UnCapsTSR and competing methods using qualitative and non-reference quantitative measures, supported by curated datasets and controlled training details.
- Evaluation compares UnCapsTSR with leading super-resolution techniques using qualitative patches and non-reference quantitative metrics.The analysis focuses on an upscaling factor of ×4 and includes statistical significance testing.
- Datasets and training details: The curated Kvasir SR dataset contains 10,000 training, 550 validation, and 1,000 testing images resized to 280 × 280 pixels.The original dataset contains 47,236 RGB images, while an additional 10,000 conventional endoscopy images support unpaired training.
- Datasets and training details: UnCapsTSR and competing methods are trained on the newly edited Kvasir dataset with an upscaling factor of ×4.Training uses 100K iterations, 64 × 64 random crops, augmentation, and Adam optimization.
- EndoQM degradation experiments: Figures 4 and 5 apply low-pass filtering, ×4 bicubic downsampling, and noise to conventional and WCE Kvasir images for EndoQM evaluation.The conventional and WCE examples begin at 832 × 832 and 280 × 280 pixels, respectively.
5.2. Usability of Endoscopy Quality Metric (EndoQM)
The experiments test whether EndoQM detects degradations in conventional and capsule endoscopic images and supports domain-specific quality assessment.
- EndoQM is evaluated on conventional and capsule images subjected to controlled degradations to assess its suitability for endoscopic image quality measurement.The experiments use the degradation pipelines shown in Figures 4 and 5.
- Downsampled images receive higher EndoQM scores, indicating that the metric captures degradation caused by ×4 downsampling.Gaussian noise with mean 0 and standard deviation 10 is subsequently added to model sensor noise and transmission artifacts.
- The validation supports EndoQM as sensitive to clinically relevant degradations in WCE and conventional endoscopy.The authors distinguish it from generic reference-less metrics that may not align with endoscopic imaging characteristics.
5.3. Performance Analysis on Newly Derived Kvasir Dataset
On the newly derived Kvasir dataset, UnCapsTSR is compared with six unsupervised SR methods through qualitative images and non-reference metrics.
- Dataset and evaluation: The Kvasir SR test set contains 1,000 LR samples at 280 × 280 pixels for evaluating the proposed unsupervised transformer-based model.
- Qualitative Analysis: UnCapsTSR qualitatively enhances WCE image quality, supporting the diagnostic importance of reconstructed visual detail.The cited discussion refers to the comparison in Figure 6.
- Qualitative Analysis: The qualitative comparison covers UnCapsTSR, SRResCGAN, DUSGAN, dSRVAE, ZSSR, DASR, and MDASR for ×4 upscaling.
- Qualitative Analysis: SRResCGAN and DUSGAN produce poor detail reconstruction, while dSRVAE, ZSSR, and DASR show smoothing, noise, or inadequate intricate-detail preservation.
- Quantitative Analysis: UnCapsTSR achieves the lowest EndoQM scores and the best NIQE result among competing methods, while outperforming alternatives across the reported metrics.Lower BRISQUE, NIQE, PIQE, and EndoQM values indicate better quality in the evaluation.
5.4. Performance analysis on KID dataset
External KID and GIANA evaluations assess whether UnCapsTSR generalizes beyond the training data, with qualitative and quantitative comparisons against state-of-the-art methods.
- Evaluation setting: The KID dataset is not presented during training and is used to evaluate generalization at ×4 upscaling.
- Qualitative Analysis: On KID, UnCapsTSR preserves vascular details and lesion textures while minimizing artifacts compared with competing methods.MDASR over-smooths critical regions, whereas other methods struggle with capillary networks or introduce artifacts.
- Quantitative Analysis: KID quantitative analysis reports significant improvements across all metrics for the proposed model.UnCapsTSR attains the lowest PIQE and BRISQUE scores, indicating minimal perceptual distortions and stronger structural coherence.
5.5. Performance analysis on GIANA dataset
On GIANA, UnCapsTSR is evaluated for generalization beyond its training data and is reported to outperform existing methods quantitatively and qualitatively.
- UnCapsTSR was evaluated on GIANA to assess generalization beyond the original training set.
- UnCapsTSR reconstructed sharper lesion boundaries while preserving natural texture patterns, unlike methods showing blurring, texture loss, or over-smoothing.
- UnCapsTSR achieved better results than existing methods across all evaluated GIANA metrics.
5.6. Statistical Analysis
The study uses multiple statistical tests to assess whether UnCapsTSR's quality-metric improvements are reliable and statistically distinguishable from baseline variation.
- ANOVA evaluated BRISQUE, PIQE, NIQE, and EndoQM results at a 95% confidence interval.
- Two-tailed Z-tests examined whether UnCapsTSR's improvements over baseline models could be attributed to random variation.
- Two-sample K–S tests compared UnCapsTSR with each baseline across BRISQUE, PIQE, NIQE, and EndoQM.
5.7. Ablation Study
The ablation study examines how loss functions and network configurations affect UnCapsTSR across three datasets using four reference-less quality metrics.
- The ablation study evaluates different loss functions and network configurations across Kvasir, KID, and GIANA datasets.
- BRISQUE, PIQE, NIQE, and EndoQM are used to assess ablation variants without reference images.
6. Conclusion
UnCapsTSR enhances low-resolution WCE images through unsupervised transformer-based GAN training without genuine LR-HR pairs or explicit degradation learning. The study adds BTV loss, cross-dataset evaluation, EndoQM, and a curated Kvasir dataset.
- UnCapsTSR enhances low-resolution WCE data through unsupervised training without genuine LR-HR paired datasets.
- The method uses a transformer within a GAN framework with direct LR-to-SR domain transfer and no explicit degradation learning.
- Bilateral Total Variation loss is used alongside perceptual GAN loss to improve spatial continuity in the final SR image.
- Generalizability is assessed on datasets excluded from training using comparisons with unsupervised SR approaches.
- The study introduces a domain-specific reference-less quality index and a curated Kvasir dataset for WCE SR research.