Source-linked AI summary
CogSENet: Blind Image Deblurring with Blur-Conditioned Semantic Routing and Explicit Frequency Fusion
Pan Wang, Yihao Hu, Xiujin Liu
TL;DR
Blind image deblurring must recover coherent details from complex, spatially varying degradations, but existing methods often lack semantic organization and explicit frequency separation. CogSENet combines content-adaptive state-space modeling, frequency-aware refinement, and blur-semantic modulation, achieving state-of-the-art restoration quality with favorable efficiency.
Problem
Existing blind deblurring methods often treat restoration as static pixel regression, overlooking semantic organization, spatially varying degradation, and explicit separation of structural frequencies.
Method
CogSENet combines semantic-routed state-space modeling, wavelet-based high/low-frequency refinement, and continuous blur-field modulation with frozen CLIP priors.
Results
CogSENet achieves state-of-the-art restoration quality with favorable memory and runtime efficiency, while full SDSSM reaches 34.21 dB on GoPro.
Takeaways & Limitations
The framework supports efficient, content-adaptive restoration that prioritizes heavily blurred semantic regions and explicitly refines complementary frequency components.
Takeaways & Limitations
The frozen CLIP priors can fail under extreme motion blur, causing nearly uniform attention, missed local motions, residual blur, and vertical artifacts.
Abstract
from arXiv · showhide
Blind image deblurring demands the recovery of high-fidelity details and coherent structures from complex, unknown degradations. Current blind image deblurring methods struggle with real-world, spatially varying degradations, and lack the semantic awareness necessary to reliably differentiate valid textures from artifacts. To bridge this gap, we propose CogSENet, a dynamic, semantic-aligned reconstruction framework inspired by the eagle's visual system. By mimicking the eagle's active saccadic scanning, we devise a Semantic-Driven State Space Module (SDSSM) with semantic-aware token regrouping via differentiable routing, enabling prompt-conditioned long-range dependency modeling. To ensure physically interpretable recovery of textures and structures, a BiFreqFusionBlock (BFFB) mirrors functional differentiation of the eagle's retina by decomposing features into high and low frequencies using wavelet transforms. Finally, we estimate a continuous Blur Field (CBF) from blur image and fuse it with CLIP semantic priors to modulate the deepest latent features, emulating focal adaptation and enabling adaptive restoration under spatially non-uniform blur. Extensive experiments demonstrate that CogSENetoutperforms state-of-the-art deblurring methods in both visual quality and structural fidelity with fewer parameters, while also performing favorably on dehazing, deraining, and denoising tasks.
1 Introduction
CogSENet reframes blind image deblurring as dynamic, semantically aligned reconstruction for unknown, complex, and spatially varying blur. Its framework combines semantic-driven routing, explicit frequency disentanglement, continuous blur-field conditioning, and frozen CLIP priors.
- Problem: Blind image deblurring must recover high-fidelity details and coherent structures despite unknown and complex blur.
- Motivation: CogSENet replaces static pixel-mapping regression with dynamic, semantically aligned reconstruction to distinguish valid textures from artifacts.
- Contributions: CogSENet estimates a continuous blur field and jointly uses frozen CLIP semantic priors to condition deep features under complex non-uniform blur.
- Contributions: SDSSM uses non-causal token allocation and semantic-driven routing for prompt-modulated selective scanning over semantically regrouped tokens.
- Contributions: BFFB applies wavelet decomposition with specialized high- and low-frequency branches for physically interpretable recovery of textures and structures.
- Result: Compared with existing SSM-based methods, CogSENet restores blind images with fewer parameters while outperforming state-of-the-art methods.
2 Related works
CNNs have been the dominant paradigm for low-level image restoration because of their locality bias and efficient computation. Related designs evolved from encoder–decoder and multi-scale architectures toward residual learning and attention for improved texture recovery.
- CNN-based image restoration methods: CNN-based restoration methods dominate low-level image restoration through strong locality bias and efficient computation.The passage identifies CNNs as the dominant paradigm in recent years.
- CNN-based image restoration methods: Early CNN approaches used encoder–decoder or multi-scale architectures to enlarge receptive fields.These architectures were introduced to address limited receptive fields in restoration models.
- CNN-based image restoration methods: Later CNN designs incorporated residual learning and attention to improve texture recovery.The passage describes residual learning and attention as subsequent design developments.
3 Method
CogSENet is a hierarchical encoder-decoder restoration network built from CogSF blocks that combine semantic long-range modeling with explicit frequency-aware refinement. It further modulates bottleneck features using continuous blur-field and CLIP semantic priors for spatially non-uniform degradations.
- Overall architecture: CogSENet uses a three-level symmetric encoder-decoder with stacked CogSF blocks, overlap-convolution shallow features, and lateral skip connections for multiscale restoration.Each level changes spatial resolution and channel width across scales, while skip connections preserve fine spatial details.
- CogSF block: Each CogSF sequentially applies SDSSM for structured long-range context aggregation and BFFB for complementary frequency-component refinement.BFFB is instantiated as FullBFFB or the more efficient LightBFFB approximation.
- Semantic-Driven State-Space Module: SDSSM uses differentiable Gumbel-Softmax routing to regroup tokens by semantic clusters and injects prompt-conditioned dynamics into selective state-space scanning.Detached sorting indices prevent gradient truncation, while continuous prompts preserve end-to-end differentiability and alternating spatial flips enable global perception.
- Bi-Frequency Fusion Block: BFFB combines orthogonal wavelet high/low decomposition with Fourier-domain spectral filtering, then uses a learnable scalar β for adaptive residual fusion.The wavelet pathway models global structures and fine details separately, while spectral filtering suppresses artifacts and recovers textures.
- Blur-conditioned semantic modulation: CogSENet applies a Continuous Blur Field and CLIP semantics at the deepest bottleneck to jointly modulate latent features under spatially non-uniform degradations.A lightweight CNN predicts a bounded displacement field, which is resized, embedded, concatenated with bottleneck features, and fused into blur-aware descriptors; CLIP-based cosine matching supplies semantic attention.
4 Experiments
CogSENet is evaluated across deblurring benchmarks and related restoration tasks, achieving strong accuracy–efficiency trade-offs and favorable cross-dataset generalization. Ablations attribute gains to semantic routing, frequency-aware fusion, and joint physical-semantic modulation, while extreme motion blur remains a limitation.
- Dataset and Evaluation: Experiments use GoPro, HIDE, RealBlur-R, and RealBlur-J for deblurring, alongside deraining, dehazing, and low-light enhancement benchmarks.Training proceeds in three stages with progressively larger patches and smaller batches, using cosine annealing throughout.
- Qualitative Results: Qualitative comparisons show cleaner edges, sharper local textures, clearer character boundaries, and more faithful structures than competing methods on GoPro and RealBlur.Competing methods often retain residual blur or over-smoothed details around fine structures, high-frequency regions, characters, and contours.
- Deblurring Results: 34.72 dB PSNR and 0.9744 SSIM on GoPro are achieved with only 8.9M parameters, exceeding FFTformer by 0.51 dB and EVSSM by 0.21 dB.The comparison reports 34.21 dB for FFTformer and 34.51 dB for EVSSM, with 16.6M/17.1M parameters respectively.
- Deblurring Results: 32.18 dB PSNR and 0.9514 SSIM are obtained when training on GoPro and testing on HIDE, while RealBlur-R and RealBlur-J reach 41.83/0.9799 and 34.54/0.9473.Our+ provides further gains under cross-dataset testing and remains consistently stronger on both real-world benchmarks.
- Cross-Task Results: On Rain100H, the model achieves the highest SSIM, trails the best PSNR by 0.02 dB, and improves PSNR over Restormer by 0.69 dB.The result is presented as evidence of robustness under severe degradation and effectiveness of the frequency-aware BFFB design.
- Ablation Studies: SDSSM ablations show that removing SDSSM yields 33.95 dB, removing semantic prompts yields 34.15 dB, and removing semantic regrouping yields 34.21 dB on GoPro.Routing assigns dominant token groups to smooth or motion-dominated regions and diverse token IDs to texture-rich regions.
- Ablation Studies: Joint continuous-blur-field and CLIP-semantic modulation substantially outperforms either prior individually, directing restoration toward degraded semantic regions.The module estimates spatially continuous blur and uses high-level semantics to isolate informative structures while suppressing flat-region responses.
- Limitations: Extreme motion blur can defeat frozen CLIP semantics and blur-field estimation, producing nearly uniform attention, residual blur, and vertical artifacts.The failure occurs when severe blur destroys distinct semantic cues and complex local motions are missed.
5 Conclusion
CogSENet is an efficient region- and frequency-aware image deblurring framework that combines content-adaptive long-range modeling, explicit frequency refinement, and physical-semantic modulation for spatially non-uniform degradations.
- 5 Conclusion: SDSSM enables linear-complexity, content-adaptive long-range modeling for image deblurring.The module is designed to model dependencies while adapting to image content.
- 5 Conclusion: BFFB explicitly decouples and refines high- and low-frequency components.This frequency-aware design separates the components before refinement.
- 5 Conclusion: A physical-semantic joint modulation mechanism couples a continuous blur field with frozen CLIP priors to address spatially non-uniform degradations.The mechanism combines physical blur information with semantic priors for adaptive restoration.