PaperlayerView exact revision on arXiv

Source-linked AI summary

CEM-TUDASR: Computationally efficient multi-modality transformer based unsupervised domain adaptive super-resolution approach

Anjali Sarvaiya, Jay Kadel, Kishor Upla, Kiran Raja

arXiv:2609.11201v1cs.CV

TL;DR

WCE image degradation limits visibility of diagnostically important structures, while paired LR-HR clinical training data are unavailable. CEM-TUDASR addresses this with an unsupervised, domain-adaptive Transformer-based super-resolution framework using attention modules and GAN-based reconstruction. It consistently outperforms existing unsupervised SR methods across clinical datasets and maintains computational efficiency, although extremely fine anatomical details and anatomically faithful textures remain challenging under severe degradation.

  • Problem

    WCE produces degraded images, and effective super-resolution is limited by domain differences and the absence of precisely aligned LR-HR training pairs.

  • Method

    CEM-TUDASR uses unsupervised domain-adaptive Transformer-based super-resolution with learned degradation, DABs, FABs, and GAN-based reconstruction.

  • Results

    CEM-TUDASR consistently outperforms state-of-the-art unsupervised SR methods on curated Kvasir, KID, and GIANA evaluations, with cross-domain testing showing generalization to retinal images.

  • Takeaways & Limitations

    The framework provides a computationally efficient foundation for unsupervised, domain-adaptive medical-image super-resolution and potential integration into resource-constrained clinical workflows.

  • Takeaways & Limitations

    Under severe degradation, extremely fine anatomical structures remain difficult to reconstruct, and GAN-based outputs may contain visually plausible details that do not match underlying anatomy.

Abstract

from arXiv · show

Wireless Capsule Endoscopy (WCE) enables non-invasive visualization of the gastrointestinal tract, but its miniaturized optics, sensor limitations, and wireless transmission constraints result in low-resolution images with reduced visibility of diagnostically important structures. This paper proposes CEM-TUDASR, a computationally efficient unsupervised Transformer-based super-resolution framework for WCE image enhancement without paired low-resolution (LR) and high-resolution (HR) training data. A domain-adaptive degradation network synthesizes realistic WCE-like LR images from HR conventional endoscopy images, reducing the domain gap and enabling effective unpaired learning. The SR generator integrates Deep Attention Blocks (DABs) and a Fusion Attention Block (FAB) to capture long-range contextual dependencies and fine local structures while preserving perceptual and structural fidelity. The model is trained on a curated dataset derived from Kvasir Capsule and evaluated on KID and GIANA for cross-dataset generalization. No-reference quality metrics, including BRISQUE, PIQE, NIQE, and the domain-specific EndoQM, show that CEM-TUDASR consistently outperforms existing unsupervised SR methods. Qualitative results further demonstrate improved restoration of mucosal textures, vascular patterns, and clinically relevant anatomical details. Cross-domain experiments on retinal images additionally demonstrate the adaptability of the framework. With only 2.67 million parameters and 169.94 GFLOPs, CEM-TUDASR achieves high-quality reconstruction while maintaining computational efficiency, making it suitable for resource-constrained clinical and embedded endoscopic applications.

1. Introduction

WCE offers non-invasive visualization but produces degraded low-resolution images, motivating an efficient unsupervised super-resolution framework that preserves clinically relevant details without paired LR-HR data.

  • 1. Introduction: WCE enables non-invasive visualization of the gastrointestinal tract, but hardware constraints produce noisy, degraded low-resolution images that make colorectal lesion identification challenging.These limitations can reduce diagnostic accuracy and increase the likelihood of missed diagnoses.
  • CEM-TUDASR, Aim, Novelty, and Objectives: The method combines degradation-aware domain adaptation with transformer context modeling, DABs, efficient attention, and FABs to reconstruct realistic and structurally consistent images.FAB integrates bicubic-upsampled inputs with learned high-frequency features, while the GAN formulation encourages sharper textures and structural fidelity.
  • CEM-TUDASR, Aim, Novelty, and Objectives: CEM-TUDASR targets ×4 WCE super-resolution without paired LR-HR training data, using unsupervised transformer-based learning for domain-specific image enhancement.The framework is designed for perceptually consistent reconstruction of WCE images.
  • CEM-TUDASR, Aim, Novelty, and Objectives: The framework is intended to preserve mucosal textures, vascular patterns, and structural consistency while maintaining computational efficiency for clinical and embedded deployment.Its stated design goal is to balance reconstruction quality with reduced parameters and FLOPs in resource-limited environments.
  • CEM-TUDASR, Aim, Novelty, and Objectives: CEM-TUDASR is evaluated on a curated Kvasir dataset and external KID and GIANA datasets, with additional emphasis on cross-domain generalization.The external datasets are excluded from training to assess generalization capability.

2. Related Work

Prior SR methods often depend on paired data or predefined degradations, limiting their suitability for realistic WCE enhancement and cross-domain use.

  • Unsupervised SR avoids reliance on accurately aligned LR–HR pairs, which are difficult to acquire in medical imaging.
  • Predefined degradations such as bicubic downsampling may not represent complex, device-specific endoscopic degradations.
  • CEM-TUDASR addresses domain generalization through feature-level adversarial alignment between conventional endoscopy and WCE images.
  • Transformer-based modules are used to capture long-range dependencies that conventional CNN receptive fields model less effectively.
  • The framework uses Fusion Attention to combine bicubic-upsampled inputs with learned high-frequency features for fine anatomical detail recovery.

3. Proposed Framework:CEM-TUDASR

CEM-TUDASR combines domain-adaptive degradation with a Transformer-based generator for unpaired ×4 WCE super-resolution, using attention modules and adversarially trained losses.

  • CEM-TUDASR performs ×4 WCE upscaling without paired training data, extending the earlier TUDASR framework.
  • The degradation model maps HR conventional endoscopy images to synthetic LR images that mimic real WCE degradations.
  • The generator combines feature extraction, Deep Attention Blocks, Fusion Attention, and upsampling to reconstruct HR outputs.
  • Deep Attention Blocks model local textures and broader anatomical context through convolutional branches and Transformer attention.
  • The Fusion Attention Block integrates learned high-frequency features with bicubic-upsampled LR input using channel and spatial attention.
  • The generator uses weighted content and adversarial losses, while the discriminator applies LSGAN training to distinguish SR from HR images.

4. Experimental Analysis

The evaluation compares CEM-TUDASR with unsupervised SR methods across WCE datasets, cross-domain retinal images, and architectural ablations.

  • The study evaluates ×4 super-resolution using qualitative comparisons and quantitative analyses against state-of-the-art unsupervised SR techniques.
  • KID and GIANA are used to assess generalization across publicly available WCE datasets.
  • Retinal-image experiments test whether the framework adapts beyond capsule endoscopy imagery.
  • ANOVA tests statistically assess observed improvements, while ablations examine architectural components and loss configurations.
  • The curated Kvasir-derived training set contains 10,000 training, 550 validation, and 1,000 testing images after preprocessing.

4.2. Endoscopy Quality Metric (EndoQM)

EndoQM is a modality-specific no-reference metric designed to evaluate perceptual quality in super-resolved endoscopic images without HR references.

  • EndoQM adapts the NIQE framework using curated endoscopic images to capture modality-specific quality characteristics.
  • The metric addresses the limited direct applicability of natural-image no-reference metrics to endoscopic texture, noise, and color distributions.
  • EndoQM evaluates high-frequency details, structural consistency, and texture fidelity without requiring reference HR images.

4.3. Qualitative Analysis

Qualitative comparisons across Kvasir-derived, KID, and GIANA datasets show that CEM-TUDASR restores clearer textures and structures than existing unsupervised SR methods.

  • CEM-TUDASR provides sharper, more color-consistent reconstructions than competing methods in qualitative comparisons on WCE datasets.The comparisons cover the newly curated Kvasir dataset, KID, and GIANA at ×4 upscaling.
  • On the Kvasir-derived dataset, competing methods produce oversmoothing, distortions, color inaccuracies, or missing fine anatomical details.
  • On KID, CEM-TUDASR preserves high-frequency textures, vascular topology, lesion boundaries, and chromatic fidelity with minimal artifacts.
  • Baseline outputs on KID and GIANA commonly blur mucosal textures, vascular features, or lesion boundaries, limiting diagnostic clarity.

4.4. Quantitative Analysis

Quantitative evaluation uses no-reference perceptual metrics across Kvasir, KID, and GIANA, with CEM-TUDASR generally achieving the strongest reported quality and structural-coherence results.

  • CEM-TUDASR achieves the lowest EndoQM scores on Kvasir, KID, and GIANA, indicating stronger domain-specific perceptual quality.
  • On KID, CEM-TUDASR records significantly lower BRISQUE, PIQE, and NIQE scores than competing methods and the lowest EndoQM score.
  • On GIANA, CEM-TUDASR consistently outperforms existing methods across evaluated no-reference metrics, including the lowest NIQE and EndoQM scores.

4.5. Cross Domain Adaptability Analysis

Cross-domain experiments indicate that CEM-TUDASR transfers effectively from gastrointestinal endoscopy to retinal images without retinal training data.

  • The retinal results demonstrate qualitative adaptability beyond gastrointestinal endoscopy across different anatomical and acquisition conditions.
  • Without access to retinal training data, CEM-TUDASR enhances retinal vascular clarity and edge sharpness.The model recovers fine structures including microaneurysms and vessel branches.

4.6. Statistical Analysis

Statistical analysis evaluates metric reliability using ANOVA and 95% confidence intervals, with EndoQM providing the strongest reported clinical-quality validation.

  • ANOVA and confidence-interval analysis support the method’s reported reliability, robustness, and practical suitability for WCE imaging.
  • CEM-TUDASR achieves the lowest mean EndoQM score and narrowest confidence interval among the evaluated methods.EndoQM is tailored to endoscopic image assessment and reflects diagnostically significant textures and vascular structures.

4.7. Ablation Study

Ablation studies across datasets and no-reference metrics show that attention modules, pooling choices, DAB depth, and loss weighting materially affect reconstruction quality and stability.

  • Efficient Spatial Attention: Removing ESA reduces edge clarity and anatomical detail, whereas including it consistently lowers BRISQUE, PIQE, and NIQE scores and improves EndoQM across datasets.ESA emphasizes diagnostically relevant spatial features while suppressing irrelevant background information.
  • Fusion Attention Block: FAB improves perceptual fidelity, structural continuity, and edge sharpness over simple feature addition or concatenation, with better BRISQUE, PIQE, NIQE, and EndoQM results.The block adaptively fuses bicubic-upsampled inputs with learned high-frequency features using channel and spatial attention.
  • Pooling in FAB: Average pooling produces softer, less sharp reconstructions than max pooling, with higher BRISQUE, PIQE, and EndoQM scores.Max pooling better highlights salient lesion boundaries and vascular structures.
  • Deep Attention Blocks: An empirically optimized number of DABs balances model complexity and reconstruction quality better than under- or over-parameterized alternatives.The ablation compares fewer and more DABs against the proposed architecture.
  • Loss weighting: Prioritizing content loss over adversarial loss (λ1 > λ2) yields the lowest BRISQUE, PIQE, NIQE, and EndoQM scores, whereas λ1 < λ2 produces artifact-prone outputs.Equal weighting preserves some textures but lacks sharpness, while excessive adversarial influence destabilizes reconstruction.
  • Convergence analysis: Generator and discriminator losses reach a steady equilibrium during training, indicating stable adversarial optimization without mode collapse.The discriminator loss decreases initially and then stabilizes, while the generator loss converges.

4.8. Computational Complexity

CEM-TUDASR uses a lightweight architecture to maintain reconstruction quality while substantially reducing parameter count and inference computation relative to unsupervised SR baselines.

  • Model comparison: 2.67M parameters and 169.94 GFLOPs make CEM-TUDASR substantially more efficient than DASR, dSRVAE, and BSRGAN.DASR uses 11.24M parameters and 470.83 GFLOPs, dSRVAE uses 7.96M and 689.95 GFLOPs, and BSRGAN uses 16.70M and 1405.57 GFLOPs.
  • Architecture: The lightweight design combines feature extraction, DABs, EA, ESA, upsampling, and FAB modules to achieve a low computational footprint.The layer-wise analysis describes the architectural and computational contributions of these major components.
  • Deployment relevance: The reported efficiency supports real-time deployment in portable and embedded medical imaging systems without compromising diagnostic structural fidelity.The deployment scope includes resource-constrained clinical applications such as WCE and point-of-care retinal screening.

4.9. Discussion and Limitations

The framework remains constrained by severe image degradation, imperfect degradation modeling, transformer complexity, and the risk of perceptually plausible but anatomically inaccurate textures.

  • Reconstruction limits: Recovering subtle vascular patterns, low-contrast mucosal textures, and small lesion boundaries remains difficult under motion blur, noise, uneven illumination, and compression artifacts.These conditions make high-frequency anatomical reconstruction inherently ill-posed.
  • Domain limitations: Generalization depends on whether the learned degradation network captures device-, protocol-, patient-, and illumination-specific real-world degradation patterns.The synthetic WCE-like degradation may not represent all complex acquisition conditions.
  • Model risks: Transformer modules add computational complexity, while GAN training can trade anatomical fidelity for over-enhanced or visually plausible details.Such deviations may influence clinical interpretation and diagnostic confidence.

5. Conclusion

CEM-TUDASR provides unsupervised, domain-adaptive super-resolution for WCE without paired LR-HR data, combining attention mechanisms with computational efficiency. It outperforms unsupervised SR methods across evaluated medical datasets and generalizes to retinal images, while severe degradation and texture-faithfulness challenges remain.

  • Contribution: CEM-TUDASR addresses unpaired WCE super-resolution through domain-specific degradation modeling and efficient EA, ESA, and FAB attention mechanisms.The framework is evaluated on the curated Kvasir SR dataset and external KID and GIANA datasets.
  • Findings: The model consistently outperforms state-of-the-art unsupervised SR methods in quantitative metrics and perceptual quality, with cross-domain retinal testing demonstrating generalization.Its reduced parameter and FLOP count supports real-time and resource-constrained medical deployment.
  • Future directions: Future work must improve reconstruction under complex degradations, prevent artificial textures, exploit temporal WCE information, and incorporate task-driven optimization.These directions target fine-detail recovery, frame consistency, and clinically significant regions.
Loading 2609.11201v1…