Source-linked AI summary
It Takes Two: A Duet of Periodicity and Directionality for Burst Flicker Removal
Lishen Qu, Shihao Zhou, Jie Liang, Hui Zeng, Lei Zhang, Jufeng Yang
TL;DR
Flicker is a structured artifact in short-exposure photography, but generic restoration methods do not model its spatial-temporal patterns effectively. Flickerformer models flicker periodicity and directionality with dedicated fusion, autocorrelation, and wavelet modules, achieving the best reported benchmark performance with fewer parameters.
Problem
Flicker caused by unstable AC illumination complicates short-exposure imaging, while generic restoration frameworks do not account for its structured spatial-temporal patterns.
Method
Flickerformer uses phase-based fusion and autocorrelation to model flicker periodicity, plus wavelet-based directional attention to exploit directional high-frequency information during restoration.
Results
Flickerformer achieves the best performance across PSNR, SSIM, and LPIPS, with 31.226 dB average PSNR, exceeding the second-best method by +0.580 dB using 19.70% of the parameters.
Takeaways & Limitations
The experiments support incorporating flicker-specific periodicity and directionality priors into burst restoration for high-quality removal without ghosting artifacts.
Takeaways & Limitations
The method relies on the assumption that flicker distribution primarily lies in the phase spectrum.
Abstract
from arXiv · showhide
Flicker artifacts, arising from unstable illumination and row-wise exposure inconsistencies, pose a significant challenge in short-exposure photography, severely degrading image quality. Unlike typical artifacts, e.g., noise and low-light, flicker is a structured degradation with specific spatial-temporal patterns, which are not accounted for in current generic restoration frameworks, leading to suboptimal flicker suppression and ghosting artifacts. In this work, we reveal that flicker artifacts exhibit two intrinsic characteristics, periodicity and directionality, and propose Flickerformer, a transformer-based architecture that effectively removes flicker without introducing ghosting. Specifically, Flickerformer comprises three key components: a phase-based fusion module (PFM), an autocorrelation feed-forward network (AFFN), and a wavelet-based directional attention module (WDAM). Based on the periodicity, PFM performs inter-frame phase correlation to adaptively aggregate burst features, while AFFN exploits intra-frame structural regularities through autocorrelation, jointly enhancing the network's ability to perceive spatially recurring patterns. Moreover, motivated by the directionality of flicker artifacts, WDAM leverages high-frequency variations in the wavelet domain to guide the restoration of low-frequency dark regions, yielding precise localization of flicker artifacts. Extensive experiments demonstrate that Flickerformer outperforms state-of-the-art approaches in both quantitative metrics and visual quality. The source code is available at https://github.com/qulishen/Flickerformer.
1. Introduction
Flicker is a structured artifact caused by periodic illumination and rolling-shutter exposure differences, while existing approaches remain limited. Flickerformer embeds flicker’s periodicity and directionality into specialized modules for burst restoration without ghosting.
- Motivation: Alternating-current illumination oscillations and rolling-shutter exposure differences produce structured flicker artifacts in short-exposure images.Short exposures capture incomplete illumination cycles, while line-by-line sensor exposure creates row-wise inconsistencies.
- Flicker priors: Phase swapping reveals that phase encodes flicker’s spatial distribution, motivating periodicity-aware fusion and autocorrelation modules.PFM models inter-frame similarity, while AFFN captures intra-frame periodic structures.
- Existing limitations: Traditional pattern-matching, brightness-estimation, hardware, and learning-based methods have not fully addressed burst flicker removal.Generic restoration networks are not specifically designed for flicker’s intrinsic characteristics.
- Flicker priors: Rolling-shutter flicker forms directional stripes and frequency variations, motivating WDAM’s wavelet-guided restoration of dark regions.WDAM uses orientation-specific high-frequency subbands to guide attention in the low-frequency branch.
- Contribution: Flickerformer combines PFM, AFFN, and WDAM in a transformer framework that removes burst flicker without ghosting and outperforms prior methods quantitatively and visually.The design jointly exploits periodicity and directionality for restoration.
2. Related Work
Prior burst-restoration and flicker-removal methods rely on generic restoration, hardware compensation, or assumptions about lighting conditions. Flickerformer instead explicitly embeds flicker priors into a transformer architecture to improve robustness and mitigate ghosting.
- Vision Transformers: Vision Transformers model long-range dependencies, while restoration variants reduce quadratic attention costs through windows, shifting, or channel-wise attention.These adaptations target practical high-resolution image restoration.
- Burst Image Restoration: Burst restoration commonly aligns and fuses multiple frames for low-light, low-resolution, and noisy photography, but such pipelines are sensitive to motion.Traditional methods often use optical flow or patch-based matching before pixel-level fusion.
- Flicker Removal: Classical flicker-removal methods adjust exposure or use lighting parameters, achieving satisfactory results in controlled settings but struggling in wild scenarios.Extending exposure can also introduce motion blur in dynamic scenes.
- Flicker Removal: Recent learning-based methods and the BurstDeflicker benchmark demonstrate progress, but generic restoration networks are not designed for burst flicker’s intrinsic characteristics.Flickerformer is presented as the first transformer architecture to explicitly embed flicker priors for this task.
3. Proposed Method
Flickerformer is a transformer-based burst-restoration architecture that models flicker periodicity with frequency-domain fusion and autocorrelation, while using wavelet directional attention to localize directional artifacts.
- Overall architecture: Flickerformer processes three burst frames with an asymmetric U-shaped encoder–decoder and predicts a residual map for the restored base frame.The frames are fused before hierarchical Transformer encoding, followed by decoder refinement and residual reconstruction.
- Periodicity modeling: PFM uses phase correlation between the base and reference-frame features to estimate frequency-component reliability and enhance reference features before fusion.The phase spectrum is used because the paper associates flicker distribution with phase information.
- Periodicity modeling: AFFN computes spatial autocorrelation through the Wiener–Khinchin theorem, amplifying repetitive structures and suppressing uncorrelated noise.Its dual-domain process balances frequency modulation and spatial reinforcement using learnable parameters.
- Directionality modeling: WDAM decomposes features with Haar wavelets, uses low-frequency features for attention, and injects horizontal and vertical high-frequency components as directional weights.The wavelet subbands represent horizontal, vertical, and diagonal components, while high-frequency variations guide directional attention.
- Directionality modeling: WDAM maintains windowed-attention representational power while reducing computational and memory costs by approximately 75%.The module combines wavelet decomposition with directional modulation before reconstructing features through inverse wavelet transformation.
4. Experiments
Experiments on BurstDeflicker compare Flickerformer with restoration models and module variants using quantitative and qualitative evaluations. Flickerformer achieves strong overall performance, while ablations support the contributions of AFFN and WDAM; large-scale light extinction remains challenging.
- Experimental Settings: Experiments use the BurstDeflicker benchmark, with all methods trained and evaluated on identical training and testing splits for fair comparison.
- Qualitative Comparison: Flickerformer provides superior visual quality by removing flicker while preserving fine textures and color; residual maps highlight pixels whose errors exceed a threshold.
- Quantitative Comparison: 31.226 dB average PSNR makes Flickerformer the best-performing method across PSNR, SSIM, and LPIPS, surpassing the second-best method by +0.580 dB with 19.70% of its parameters.
- Ablation Study: AFFN improves PSNR by about +0.265 dB over FRFN at nearly identical parameter counts and restores extinguished-light regions without motion ghosting.
- Ablation Study: WDAM gains +0.229 dB in PSNR over the best alternative at lower computational cost and more precisely localizes subtle flicker through high-frequency information.
- Limitations: Flickerformer struggles to restore regions affected by large-scale light extinction, identifying a limitation of the evaluated restoration setting.
5. Conclusion
Flickerformer removes flicker by modeling its periodicity and directionality with dedicated transformer modules. Experiments on real-world datasets report better quantitative performance and visual quality than state-of-the-art methods, while large-scale scene-wide light extinction remains a limitation.
- Flickerformer leverages flicker periodicity and directionality through PFM, AFFN, and WDAM for burst flicker removal.
- PFM uses phase correlation to aggregate information across frames, while AFFN enhances recurrent structural cues through frequency-domain autocorrelation.
- WDAM uses directional high-frequency information to guide restoration of low-frequency regions and improve flicker removal.
- Experiments on real-world datasets show that Flickerformer surpasses state-of-the-art methods in quantitative performance and visual quality.
- Limitations: Flickerformer struggles when clean regions across multiple flickering frames fail to cover the entire scene.