Source-linked AI summary
Dual-Branch Remote Sensing Infrared Image Super-Resolution
Xining Ge, Gengjia Chang, Weijun Yuan, Zhan Li, Zhanglu Chen, Boyang Yao, Yihang Chen, Yifan Deng, Shuhong Liu
TL;DR
Infrared super-resolution must recover contours and stable scene structure from weakly textured thermal inputs. The paper combines HAT-L and MambaIRv2-L with branch-specific test-time refinements and fixed equal-weight fusion, and reports fused outputs outperforming either branch on reproducible thermal evaluation.
Problem
Thermal imagery is weakly textured and sensitive to unstable local sharpening, making faithful contours, scene layout, and radiometric stability important restoration targets.
Method
A dual-branch system combines HAT-L with test-time local conversion and MambaIRv2-L with eight-way self-ensemble, followed by fixed equal-weight image-space fusion.
Results
The fused output outperforms either single branch in PSNR, SSIM, and overall Score on 12 synthetic times-four Caltech Aerial RGB-Thermal samples.
Takeaways & Limitations
Pairing locally strong transformer restoration with globally stable state-space modeling is an effective strategy for infrared super-resolution.
Abstract
from arXiv · showhide
Remote sensing infrared image super-resolution aims to recover sharper thermal observations from low-resolution inputs while preserving target contours, scene layout, and radiometric stability. Unlike visible-image super-resolution, thermal imagery is weakly textured and more sensitive to unstable local sharpening, which makes complementary local and global modeling especially important. This paper presents our solution to the NTIRE 2026 Infrared Image Super-Resolution Challenge, a dual-branch system that combines a HAT-L branch and a MambaIRv2-L branch. The inference pipeline applies test-time local conversion on HAT, eight-way self-ensemble on MambaIRv2, and fixed equal-weight image-space fusion. We report both the official challenge score and a reproducible evaluation on 12 synthetic times-four thermal samples derived from Caltech Aerial RGB-Thermal, on which the fused output outperforms either single branch in PSNR, SSIM, and the overall Score. The results suggest that infrared super-resolution benefits from explicit complementarity between locally strong transformer restoration and globally stable state-space modeling.
1. Introduction
Infrared super-resolution must recover faithful structure because thermal imagery is weakly textured and sensitive to unstable sharpening. The paper addresses this with complementary local and global restoration branches, evaluated on challenge and reproducible benchmarks.
- Thermal imagery has weak texture, so boundary quality and faithful structural recovery can matter more than aggressive texture generation.
- Local sharpening can recover thermal contours, whereas global propagation can preserve scene-wide stability and suppress unstable hot spots.
- The proposed solution combines HAT-L and MambaIRv2-L with branch-specific test-time refinements and equal-weight fusion.
- The fused output outperforms either single branch in PSNR, SSIM, and overall Score on 12 synthetic times-four thermal samples.
2. Related Work
Related work spans transformer and state-space super-resolution, while infrared research emphasizes sparse texture, stable transitions, realistic degradation, and reproducible evaluation. Test-time enhancement and output fusion provide training-free ways to exploit complementary restoration behaviors.
- Transformer restorers emphasize local-global interaction, while state-space models strengthen long-range propagation with lightweight, scalable architectures.
- Infrared super-resolution depends more on stable region transitions than dense high-frequency detail, motivating specialized thermal reconstruction methods.
- Test-time local conversion addresses patch-training versus full-image inference discrepancy, while x8 self-ensemble can provide consistent inference gains without retraining.
3. Method
The method uses two independently optimized branches: HAT-L emphasizes local contour recovery, while MambaIRv2-L aggregates long-range context. Branch-specific test-time refinement and deterministic equal-weight fusion combine their complementary outputs.
- 3.1. Dual-Branch Framework: Both branches receive the same low-resolution infrared input and are fused only after their separate inference paths are completed.
- 3.1. Dual-Branch Framework: The framework keeps branches decoupled and uses deterministic equal-weight image-space fusion instead of feature interaction or learned fusion heads.
- 3.2. Local and Global Branches: HAT-L targets fine-grained contour recovery and small-structure enhancement, whereas MambaIRv2-L propagates information over long distances and preserves large-scale consistency.
- 3.3. Test-Time Refinement: Test-time local conversion is applied to HAT, eight-way geometric self-ensemble to MambaIRv2, and fixed equal-weight fusion merges the refined outputs.
- 3.3. Test-Time Refinement: The asymmetric refinements reflect branch complementarity: TLC addresses window-transformer mismatch, while self-ensemble stabilizes state-space prediction.
4. Experiments
Experiments evaluate the system on the official challenge benchmark and a reproducible 12-frame Caltech Aerial RGB-Thermal set, combining quantitative and qualitative comparisons. Fusion provides the strongest overall restoration while balancing local edge sharpness with global structural stability.
- 54.23 final score on the official NTIRE 2026 hidden test set.
- The reproducible evaluation uses 12 Caltech Aerial RGB-Thermal frames with bicubic ×4 downsampling, modcropping, and a 4pixel shaved border before metrics.
- Qualitative Results: The qualitative comparison uses GT, Bicubic, MambaIRv2, HAT, and Fusion columns on representative thermal synthetic ×4 samples.
- Fusion achieves the best performance across PSNR, SSIM, and the combined Score, surpassing both learned single branches and bicubic.
- Qualitative Results: Fusion retains HAT-L’s sharper local contours while inheriting MambaIRv2-L’s global radiometric consistency and structural stability.
- Qualitative Results: Faithful structural recovery is favored over aggressive high-frequency texture synthesis for infrared imagery.
5. Conclusion
The paper presents a dual-branch HAT-L and MambaIRv2-L framework with branch-specific test-time refinement and fixed equal-weight fusion. Across the official challenge benchmark and Caltech Aerial RGB-Thermal evaluation, the fused output consistently surpasses either single branch.
- The framework couples HAT-L with test-time local conversion and MambaIRv2-L with eight-way self-ensemble, then averages their outputs equally in image space.
- The fused output consistently surpasses either single branch on both evaluated benchmarks.