PaperlayerView exact revision on arXiv

Source-linked AI summary

Blind Predicting Similar Quality Map for Image Quality Assessment

Da Pan, Ping Shi, Ming Hou, Zefeng Ying, Sizhe Fu, Yuan Zhang

arXiv:1805.08493v2cs.CV

TL;DR

Blind IQA seeks to model human visual-system properties without reference images, and prior deep methods often lack explicit intermediate quality maps. BPSQM predicts pixel-level maps from distorted images using FR-IQA-derived similarity-map guidance, then pools them into scores; the authors report that it outperforms state-of-the-art BIQA methods except for DIQaM on PLCC on LIVE.

  • Problem

    Blind IQA lacks reference-image information, and prior deep methods often do not generate intermediate quality maps that represent the perceptual impact of degradations.

    Source: §I · pp. 1–2

  • Method

    BPSQM trains an FCNN with FR-IQA-derived similarity-map guidance to predict pixel-level quality maps from distorted images, then uses a deep pooling network to estimate scores.

    Source: §I · p. 2, §III · p. 3

  • Results

    BPSQM outperforms state-of-the-art BIQA methods except for DIQaM on PLCC on LIVE.

    Source: §I · p. 8

  • Takeaways & Limitations

    The authors report that pixel-level similarity quality maps can achieve good consistency with human perception when fed into a learnable pooling network.

    Source: §I · p. 2

    The authors identify improving the similarity map used to guide training and optimizing pixel-distortion prediction as future work.

    Source: §V · p. 10

Abstract

from arXiv · show

A key problem in blind image quality assessment (BIQA) is how to effectively model the properties of human visual system in a data-driven manner. In this paper, we propose a simple and efficient BIQA model based on a novel framework which consists of a fully convolutional neural network (FCNN) and a pooling network to solve this problem. In principle, FCNN is capable of predicting a pixel-by-pixel similar quality map only from a distorted image by using the intermediate similarity maps derived from conventional full-reference image quality assessment methods. The predicted pixel-by-pixel quality maps have good consistency with the distortion correlations between the reference and distorted images. Finally, a deep pooling network regresses the quality map into a score. Experiments have demonstrated that our predictions outperform many state-of-the-art BIQA methods.

I. INTRODUCTION

Blind IQA must infer human-vision-related quality without a reference image, while prior deep methods often provide no explicit perceptual interpretation. BPSQM predicts pixel-level quality maps from distorted images and pools them into image scores.

I. INTRODUCTION

  • Deep-IQA methods often treat CNNs as regression functions or feature extractors, leaving intermediate quality maps and the learning process without explicit perceptual meaning.BIECON estimates patch scores, which do not represent distortion at the pixel level.Source: §I · p. 1
  • BPSQM combines an FCNN that predicts a pixel-level quality map with a deep pooling network that regresses the map into an image score.Similarity-map labels guide the FCNN to learn local pixel-distortion features.Source: §I · p. 2
  • Guided by HVS-related similarity maps from FR-IQA, BPSQM predicts global quality maps whose distortion distribution is generally similar to FSIM's feature map.The authors describe the map as pixel-wise, unlike BIECON's patch-based map.Source: §I · p. 2
  • The paper introduces pixel-to-pixel segmentation for predicting global pixel-level similarity maps close to maps from FR-IQA.The authors also state that the predicted maps can provide intuitive local-distortion analysis to improve image-enhancement modules.Source: §I · p. 2

II. RELATED WORK

The related work contrasts FR-IQA's similarity-map-and-pooling pipeline with NR-IQA methods based on natural-image statistics, then reviews CNN-based IQA and its interpretability limits.

A. Full-reference Image Quality Assessment

  • FR-IQA methods compute HVS-related similarity maps from reference and distorted images, then pool them into quality scores.Examples include SSIM and FSIM, which use different image features to construct maps.Source: A. Full-reference Image Quality… · p. 2

B. No-reference Image Quality Assessment

  • NR-IQA methods estimate perceived quality from natural-image statistics, using approaches such as distortion-specific regression, DCT statistics, learned patch dictionaries, or Gaussian models.Examples include DIIVINE, BLIINDSII, CORNIA, and NIQE.Source: B. No-reference Image Quality Assessment · p. 2

C. Deep Image Quality Assessment

  • Deep-IQA methods commonly use CNNs for feature extraction or score prediction, limiting interpretation of intermediate results; BIECON estimates patch-score maps and regresses patch features into a score.The proposed framework's generative network predicts a quality map and its pooling network regresses that map into a score.Source: C. Deep Image Quality Assessment · pp. 2–3, Figure 2 · p. 3

III. QUALITY MAP PREDICTION

The paper contrasts direct blind score regression with FR-IQA's similarity-map-and-pooling process, then combines these ideas by predicting a quality map from a distorted image and pooling it into a score.

III. QUALITY MAP PREDICTION

  • Conventional deep NR-IQA uses a CNN regression function to map a distorted image to a subjective quality score.The authors note that this direct input-to-score process does not reveal how the network learns distortion-related features.Source: §III · p. 3
  • FR-IQA first generates a similarity index map from reference and distorted images, then applies a pooling strategy to obtain a score.The paper proposes combining FR-IQA's HVS modeling with deep NR-IQA without hand-crafted features.Source: §III · p. 3
  • The proposed approach uses a generative network to predict a pixel-level quality map from a distorted image, then a deep pooling network converts it into a score.The pooling network is described as a complicated pooling strategy.Source: §III · p. 3

A. Architecture

The framework uses U-Net to predict full-size quality maps, trains those maps against labels from FR-IQA methods, and pools one or more map types to estimate image quality.

A. Architecture

  • U-Net generates a pixel-wise map the same size as its input, using subsampling and upsampling paths with corresponding feature maps joined across paths.The generator ends with a one-channel map squashed to [0,1].Source: A. Architecture · p. 3
  • The pooling network uses five convolutional layers with max pooling, followed by two fully connected layers and a squared Euclidean loss.The input image is cropped into overlapping fixed-size patches for pooling.Source: A. Architecture · p. 4

B. Quality Map Selection

  • SSIM maps are used directly as labels; FSIM contributes separate phase-congruency and gradient-magnitude labels, while MDSI contributes gradient and chromaticity similarity maps.Source: B. Quality Map Selection · p. 4

C. Multi Types Quality Maps Fusion

  • The fusion options are one pooling stream over concatenated maps or independent pooling streams whose final convolutional layers are concatenated before fully connected layers.Source: C. Multi Types Quality Maps Fusion · p. 4

D. Regression

The pooling network regresses predicted quality maps to subjective scores using image patches, and the evaluation spans synthetic and authentic distortion datasets.

D. Regression

  • RGB patches of 144×144×3 are sampled with a 120-pixel sliding-window step, and each patch receives its distorted image’s subjective ground-truth score.The final image score averages the cropped patches.Source: D. Regression · p. 4

A. Datasets

  • The evaluation uses four synthetic and three authentic datasets, with authentic images collected from real-world cameras and lacking corresponding reference images.The datasets include LIVE, CSIQ, TID2008, TID2013, CLIVE, CID2013, and KonIQ-10k.Source: A. Datasets · pp. 4–5
  • KonIQ-10k contains 10,073 images sampled from about 4.8 million YFCC100m images, with 1.2 million ratings from 1,467 crowd workers.Source: A. Datasets · p. 5
  • Performance is assessed with PLCC and SRCC, where higher values indicate better performance; TID2013 MOS and CSIQ DMOS are scaled to LIVE’s range.TID2013 is described as the more challenging of the four synthetic datasets.Source: A. Datasets · p. 5

B. Training Method

Training first fits the generative network to similarity maps, then fixes its parameters while training the overall framework, with reference-image-separated splits for testing.

B. Training Method

  • The U-Net is pretrained on 3,000 high-quality images with four distortion types at two levels, then trained on similarity maps before its parameters are fixed for overall-framework training.The authors report that individually trained U-Net maps are closer to similarity-index maps than maps from joint training.Source: B. Training Method · p. 5
  • Training uses ADAM, separate initial learning rates for the generative and pooling networks, and weight decay; only horizontal flips expand training data.Models are trained for 100 epochs, selecting the model with the lowest validation error.Source: B. Training Method · p. 5
  • Training and testing use an 80/20 split by reference images, repeated 10 times for individual-dataset tests and 100 times for cross-dataset tests.This keeps test image content out of the training set.Source: B. Training Method · p. 5

C. Comparison of pixel distortion and patch distortion

The experiments compare pixel-based with averaged patch-based quality maps and examine whether predicted maps track distortion patterns and levels.

C. Comparison of pixel distortion and patch distortion

  • SRCC=0.928 and PLCC=0.934 for pixel-based maps (size=1), outperforming maps formed by averaging larger local patches on TID2013.The authors report that performance decreases as local patch size increases.Source: C. Comparison of pixel distortion and… · p. 5

D. Quality Map Prediction

  • Predicted maps broadly match ground-truth distortion areas: JPEG edges and HFN spread are visible, while LBWD block locations align despite some false predictions.The figure compares JPEG, HFN, and LBWD maps across SSIM, Fg, and MDSI labels and predictions.Source: Figure 4 · p. 6, D. Quality Map Prediction · p. 7
  • As spatially correlated noise or JPEG distortion increases, predicted maps darken and the pooled scores decrease.For JPEG, the blocking artifact on the sculpture area is emphasized at higher compression.Source: D. Quality Map Prediction · p. 7, Figure 5 · p. 7

E. Dependency on FR-IQA Similarity Map

The ablations favor pooling informative similarity maps over pooling distorted images directly, and show that both map labels and pooling architecture affect performance. MDSI-derived maps rank highest among the tested labels, while the proposed DPN outperforms FC2.

E. Dependency on FR-IQA Similarity Map

  • SSIM ground-truth map pooling raises SRCC from 0.637 to 0.904, and pooling the distorted image directly performs worse than Fg PM.The authors attribute the direct-image result to redundant information that does not highlight distortion patterns.Source: E. Dependency on FR-IQA Similarity Map · p. 7, Table II · p. 7

F. Similarity Map Labels Comparison

G. Effects of Pooling Network

  • DPN outperforms FC2, while the deeper ResNet alternatives do not establish that greater pooling-network depth is necessary.The authors emphasize the quality of the generated feature maps over network depth.Source: G. Effects of Pooling Network · p. 8

H. Quality Maps Fusion

BPSQM outperforms most compared BIQA methods across the evaluated datasets, with an exception on LIVE PLCC. Fusion tests favor a single pooling stream and show that combining more map types does not consistently improve performance.

H. Quality Maps Fusion

  • A single pooling stream performs better than multiple streams, consistent with U-Net quality maps being more aligned with human vision than pooling-network feature maps.The comparison evaluates alternative schemes for combining predicted quality maps.Source: H. Quality Maps Fusion · p. 8, Table V · p. 8
  • Combining more map types does not yield significant gains, and adding predicted SSIM maps to other maps causes slight performance degradation.The combination of MDSI and FSIM gradient maps performs best; the authors relate this to their different gradient operators.Source: H. Quality Maps Fusion · p. 8, Table V · p. 8

I. Performance Comparison

  • BPSQM outperforms the compared BIQA methods except DIQaM on LIVE PLCC; BPSQM-Fg-MD is competitive with some FR-IQA methods.The overall comparison covers LIVE, CSIQ, and TID2013.Source: §I · p. 8, Table VI · p. 9
  • BPSQM and BPSQM-Fg-MD perform competitively on most LIVE and TID2008 distortion types, but BIECON does better on GB, JGTE, and J2TE.BPSQM is reported as more capable than BIECON on SCN, HFN, QN, and JP2K.Source: §I · p. 9, Table VII · p. 9

J. Performance on authentic datasets

On authentic distortion datasets, BPSQM performs better than most compared methods, and its predicted maps correspond to visible noise and blur regions. The authors also report weaker CLIVE metrics and identify the limited training-image diversity as a possible constraint.

J. Performance on authentic datasets

  • On CLIVE, BPSQM-Pre obtains SRCC=0.716 and PLCC=0.721, below its synthetic-dataset performance.The authors identify dependence on the selected FR-IQA map and CLIVE's 698 training images as possible reasons.Source: J. Performance on authentic datasets · p. 9, Table VIII · p. 9
  • BPSQM performs better than most methods across the three authentic datasets, while the CLIVE examples show darker map regions over visible noise and blur.The authors describe the maps as basically consistent with human judgments on authentic distortions.Source: J. Performance on authentic datasets · p. 9, Table VIII · p. 9
  • The authors attribute CLIVE's weaker prediction partly to limited training-image content diversity, which may leave distortion characteristics underrepresented.They contrast CLIVE's 698 distorted training images with the need for broader image-content coverage.Source: J. Performance on authentic datasets · pp. 9–10

K. Cross Dataset Test

The cross-dataset test reports good performance on four distortion types shared by LIVE and TID2008. The conclusion identifies improved similarity-map guidance and more accurate pixel-distortion prediction as routes for future improvement.

K. Cross Dataset Test

  • BPSQM performs well on the shared JP2K, JPEG, WN, and BLUR distortion types in the LIVE-to-TID2008 test.The evaluation trains on LIVE and tests on TID2008 using the four common distortion types.Source: K. Cross Dataset Test · p. 10, Table IX · p. 10

V. CONCLUSION

  • The authors conclude that better similarity-index maps could improve performance, and identify more accurate pixel-distortion prediction as future work.The model generates pixel-level quality maps guided by similarity maps derived from FR-IQA methods.Source: §V · p. 10

Related papers

Loading 1805.08493v2…