Source-linked AI summary
Towards Flexible Blind JPEG Artifacts Removal
Jiaxi Jiang, Kai Zhang, Radu Timofte
TL;DR
Existing blind JPEG artifact removal methods lack adjustable quality-factor control and can fail on non-aligned double JPEG images. FBCNN predicts and embeds a quality factor for flexible restoration, while addressing compound degradations through quality-factor correction or augmented training data. Experiments across single, double, and real-world JPEG images demonstrate its flexibility, effectiveness, and generalizability.
Problem
Existing blind JPEG methods reconstruct inputs without predicting quality factors, limiting control over the trade-off between artifact removal and detail preservation; they can also fail on non-aligned double JPEG images when QF1 ≤QF2.
Method
FBCNN decouples a latent quality factor from the JPEG image, embeds it in the reconstructor through quality-factor attention, and handles double JPEG degradation through quality-factor correction or augmented training data.
Results
Experiments on single JPEG, double JPEG, and real-world JPEG images demonstrate FBCNN’s flexibility, effectiveness, and generalizability for JPEG restoration.
Takeaways & Limitations
Adjustable quality-factor control lets FBCNN balance artifact removal and detail preservation while extending blind restoration toward complex real-world JPEG degradations.
Takeaways & Limitations
Existing blind methods fail on non-aligned double JPEG images when QF1 ≤QF2, even with a one-pixel shift, motivating specialized handling.
Abstract
from arXiv · showhide
Training a single deep blind model to handle different quality factors for JPEG image artifacts removal has been attracting considerable attention due to its convenience for practical usage. However, existing deep blind methods usually directly reconstruct the image without predicting the quality factor, thus lacking the flexibility to control the output as the non-blind methods. To remedy this problem, in this paper, we propose a flexible blind convolutional neural network, namely FBCNN, that can predict the adjustable quality factor to control the trade-off between artifacts removal and details preservation. Specifically, FBCNN decouples the quality factor from the JPEG image via a decoupler module and then embeds the predicted quality factor into the subsequent reconstructor module through a quality factor attention block for flexible control. Besides, we find existing methods are prone to fail on non-aligned double JPEG images even with only a one-pixel shift, and we thus propose a double JPEG degradation model to augment the training data. Extensive experiments on single JPEG images, more general double JPEG images, and real-world JPEG images demonstrate that our proposed FBCNN achieves favorable performance against state-of-the-art methods in terms of both quantitative metrics and visual quality.
1. Introduction
JPEG artifact removal methods face a flexibility gap: many require quality-factor-specific models, while blind methods do not expose quality-factor control. FBCNN addresses this with adjustable quality-factor prediction and targets challenging double JPEG restoration.
- JPEG compression: JPEG compression uses 8 × 8 DCT blocks and a quantization table whose quality factor controls compression and information loss.The quality factor ranges from 0 to 100; lower values reduce storage size while losing more information.
- Limitations: Existing methods often train separate models for each quality factor, while blind methods provide only a deterministic reconstruction for each input.DCT-based methods also require metadata unavailable after multiple compressions, limiting practical use.
- FBCNN: FBCNN is a single pixel-domain model that predicts a latent quality factor and uses it to guide restoration without requiring image-format metadata.The predicted factor can be manually adjusted to balance artifact removal and detail preservation.
- Double JPEG restoration: FBCNN addresses non-aligned double JPEG compression, where existing blind methods can fail when block grids are misaligned and QF1 ≤QF2.The paper proposes quality-factor correction or training-data augmentation with non-aligned double JPEG images.
- Evaluation: Experiments on synthetic and real JPEG images with complex degradation settings demonstrate FBCNN’s effectiveness for practical restoration.The evaluation covers single JPEG, double JPEG, and real-world JPEG images.
2. Related Work
Prior work spans deep, transform-based, double-compression, and flexible restoration methods. These approaches improve JPEG artifact removal or accommodate degradation variation, but the cited passages motivate further handling of compound and controllable JPEG degradations.
- JPEG artifacts removal: Learning-based JPEG artifact removal progressed from early deep networks to architectures using batch normalization, residual learning, wavelets, and quantization-table priors.Quantization-table priors enabled one model to correct artifacts across quality factors and achieved state-of-the-art results.
- Double JPEG compression: Double JPEG compression research distinguishes aligned and non-aligned cases and studies their periodic characteristics in spatial and DCT domains.The area has also been used for detecting compression history in image forensics.
- Flexible image restoration: Flexible image restoration methods condition restoration on variables such as noise level, blur kernel, and degradation parameters.These approaches provide tunability for restoration settings, although the cited passage notes they cannot be directly adopted from flexible image generation.
3. Proposed Method
FBCNN is an end-to-end flexible blind JPEG restoration network that decouples quality-factor information, predicts it, and injects controllable embeddings into image reconstruction. It also addresses complex non-aligned double JPEG artifacts through quality-factor adjustment and training-data augmentation.
- Architecture: FBCNN processes JPEG images in the pixel domain through a decoupler, QF predictor, flexible controller, and image reconstructor.The decoupler separates image features from QF features, which feed reconstruction and quality-factor prediction branches.
- Quality-factor control: The QF predictor estimates compression quality, while the controller maps a selected quality factor to embeddings for reconstruction control.The controller produces modulation parameters for multiple reconstructor scales.
- Quality-factor control: QF attention applies learned affine transformations to intermediate feature maps, enabling different restored outputs for different quality-factor settings.The transformation scales and shifts feature maps using γ and β parameters.
- Training objective: The model jointly optimizes image reconstruction and quality-factor estimation with L1 losses, balancing them through λ.The total objective is L_total = L_rec + λ · L_QF.
- Double JPEG restoration: Existing blind methods fail on non-aligned double JPEG compression when QF1 ≤ QF2, while FBCNN uses quality-factor adjustment and degradation augmentation to address this case.The difficult setting involves composite artifacts from non-aligned double compression; proposed solutions modify the QF or augment training data.
4. Experiments
Experiments evaluate FBCNN on single JPEG images, flexible quality-factor control, non-aligned double compression, and real-world meme images. Results show strong blind restoration performance, while double-compression ordering and unavailable ground truth remain important evaluation boundaries.
- Single JPEG image restoration: FBCNN covers a full range of quality factors, outperforming other blind methods and moderately outperforming MWCNN on single JPEG restoration.Evaluation uses PSNR, SSIM, and PSNR-B on Classic5, LIVE1, and BSDS500.
- Color JPEG image restoration: FBCNN-C achieves better color JPEG restoration performance than QGAC on LIVE1, BSDS500, and ICB.FBCNN-C is trained on RGB channels for color restoration.
- Flexible JPEG image restoration: Different manually set quality factors produce different perceptual results, letting users balance artifact removal against detail preservation interactively.With a smaller set quality factor, artifacts around words are removed more effectively, but brick texture can become blurred.
- Double JPEG image restoration: Changing the order of double JPEG quality factors causes a significant performance drop despite JPEG-image PSNR differences generally below 0.05 dB.The reported ordering effect affects other methods and FBCNN, especially under non-aligned compression.
- Double JPEG image restoration: FBCNN-D improves performance for QF1 < QF2 by correcting the predicted quality factor, while FBCNN-A further improves QF1 < QF2 and QF1 = QF2 cases.FBCNN-A augments training with the proposed double JPEG degradation model.
- Real-world JPEG image restoration: Real meme-image experiments report no quantitative results because high-quality ground truth and reliable no-reference IQA metrics are unavailable.The study of no-reference IQA for JPEG artifact removal is left for future work.
5. Conclusions
The paper proposes FBCNN for flexible blind JPEG artifact removal and extends evaluation to non-aligned double JPEG restoration and real-world degradations. Its predicted quality factor can be adjusted to balance artifact removal and detail preservation.
- Conclusions: FBCNN decouples the quality factor from the input image and embeds it into reconstruction through a quality factor attention block.The predicted quality factor can be adjusted to control the balance between artifact removal and detail preservation.
- Conclusions: Extensive experiments on single JPEG, double JPEG, and real-world JPEG images demonstrate FBCNN’s flexibility, effectiveness, and generalizability.The conclusion covers different kinds of degraded JPEG images.