PaperlayerView exact revision on arXiv

Source-linked AI summary

Mean Deviation Similarity Index: Efficient and Reliable Full-Reference Image Quality Evaluator

Hossein Ziaei Nafchi, Atena Shahkolaei, Rachid Hedjam, Mohamed Cheriet

arXiv:1608.07433v4cs.CV

TL;DR

The paper addresses limitations in full-reference IQA by proposing MDSI, which combines new gradient and chromaticity similarities with deviation pooling. Across natural, synthetic, and digitally retouched image datasets, the index provides comparable or better quality predictions than competing metrics while remaining reliable and low-complexity.

  • Problem

    Existing gradient similarity can disagree with the human visual system, while color changes require complementary chromaticity information and pooling strategies must account for distortion spread.

  • Method

    MDSI fuses a new gradient similarity, an efficient joint chromaticity similarity map, and a general deviation-pooling formulation to compute image quality.

  • Results

    MDSI provides comparable or better quality predictions than competing state-of-the-art IQA metrics across natural, synthetic, and digitally retouched image datasets.

  • Takeaways & Limitations

    The proposed index combines effective and reliable quality prediction with low complexity and sufficient speed for real-time full-reference IQA applications.

Abstract

from arXiv · show

Applications of perceptual image quality assessment (IQA) in image and video processing, such as image acquisition, image compression, image restoration and multimedia communication, have led to the development of many IQA metrics. In this paper, a reliable full reference IQA model is proposed that utilize gradient similarity (GS), chromaticity similarity (CS), and deviation pooling (DP). By considering the shortcomings of the commonly used GS to model human visual system (HVS), a new GS is proposed through a fusion technique that is more likely to follow HVS. We propose an efficient and effective formulation to calculate the joint similarity map of two chromatic channels for the purpose of measuring color changes. In comparison with a commonly used formulation in the literature, the proposed CS map is shown to be more efficient and provide comparable or better quality predictions. Motivated by a recent work that utilizes the standard deviation pooling, a general formulation of the DP is presented in this paper and used to compute a final score from the proposed GS and CS maps. This proposed formulation of DP benefits from the Minkowski pooling and a proposed power pooling as well. The experimental results on six datasets of natural images, a synthetic dataset, and a digitally retouched dataset show that the proposed index provides comparable or better quality predictions than the most recent and competing state-of-the-art IQA metrics in the literature, it is reliable and has low complexity. The MATLAB source code of the proposed metric is available at https://www.mathworks.com/matlabcentral/fileexchange/59809.

I. INTRODUCTION

The paper presents MDSI to improve full-reference IQA by combining gradient and chromaticity similarities with deviation pooling, targeting a balance between prediction accuracy and complexity.

  • MSE is simple but often fails to correlate with human judgments of image fidelity and quality.
  • Successful IQA models typically compute a similarity map from image features and pool it into a single quality score.
  • MDSI combines gradient magnitude for structural distortions and chrominance features for color distortions before applying deviation pooling.
  • The proposed gradient similarity is designed to follow the human visual system more closely than commonly used gradient similarity maps.
  • The proposed chromaticity similarity map has lower complexity and slightly better quality predictions than previously used chromaticity maps when paired with MDSI.
  • Experiments cover six natural-image datasets, a synthetic dataset, and a digitally retouched dataset.

II. MEAN DEVIATION SIMILARITY INDEX

The proposed full-reference IQA model combines gradient and chromaticity similarity maps, then pools them using deviation pooling. Its gradient component addresses limitations of conventional gradient similarity in modeling human visual judgments of structural distortions.

  • Model overview: The model uses image gradients to measure structural distortions and chromaticity similarity to measure color distortions.These two similarity maps are combined and pooled by a deviation pooling strategy.
  • Model overview: Luminance is computed as L = 0.2989R+0.5870G+0.1140B, alongside two chromaticity channels from a Gaussian color model.
  • Gradient similarity: Prewitt gradients are computed on reference and distorted luminance channels, and gradient similarity is calculated with an SSIM-induced equation.Sobel, Scharr, and Prewitt operators perform almost the same within the proposed model.
  • Gradient similarity: Gradient similarity is widely used for measuring image distortions, with a constant C1 providing numerical stability.
  • Gradient similarity: Conventional gradient similarity can disagree with the human visual system because added and removed edges receive no distinction in the formulation.The passage notes that edge color relative to corresponding reference pixels affects human visual attention.

B. The Proposed Gradient Similarity

The proposed gradient similarity fuses reference and distorted luminance before computing additional similarity maps, incorporating image correlation into the gradient measure. The resulting maps emphasize removed edges relative to added edges, while their different numerical scales require comparison after final pooling.

  • Fusion-based gradient similarity: The fusion technique incorporates correlation between reference and distorted images into the gradient similarity computation.
  • Scale consideration: The GS and c GS maps exhibit complementary behavior.
  • Fusion-based gradient similarity: Reference and distorted luminance channels are averaged as F = 0.5 × (R + D) to form a fused image.
  • Fusion-based gradient similarity: The fused-image gradient produces two additional similarity maps, and the proposed c GS combines these maps with the original gradient-similarity computation.The fused gradient GF is not equal to the average of GR and GD.
  • HVS-oriented behavior: The proposed gradient similarity places more emphasis on removed edges than added edges, while weak edges may smooth out in the fused image.
  • Scale consideration: GS is bounded between 0 and 1, whereas c GS can have values below 0 or above 1, so comparison is performed on the final similarity map.

C. Chromaticity Similarity

The paper introduces a joint chromaticity similarity formulation that measures both chromaticity channels together, improving efficiency while preserving comparable quality predictions. Chromaticity similarity complements gradient similarity because color changes can occur without substantial structural changes.

  • Color changes can make gradient similarity inaccurate when distorted-image structure remains unchanged, motivating a separate chromaticity similarity map.
  • The proposed joint chromaticity similarity computes both chromaticity channels simultaneously and assigns them equal weight.
  • The joint formulation requires 6 multiplications, 6 summations, 1 division, and 1 shift operation, compared with 7 multiplications, 6 summations, 2 divisions, and 2 shifts for the conventional formulation.
  • Gradient and chromaticity similarity maps are combined through a weighted sum controlled by α, which adjusts their relative importance.
  • Gradient and chromaticity similarity are complementary: gradient similarity can suffice for strong structural changes but can mispredict cases where color changes dominate.

D. Deviation Pooling

The paper develops deviation pooling to summarize similarity-map variations using both distortion magnitude and spatial spread. Its generalized formulation incorporates Minkowski-style deviation and power pooling to shape the final quality score.

  • Deviation pooling reflects the HVS sensitivity to both distortion magnitude and its spread across an image.
  • Mean, Minkowski, and percentile pooling do not account for distortion spread in the same way, motivating deviation pooling for cases with different distortion distributions.
  • Similarity-map values are interpreted so that lower values indicate more severe distortions, while higher values indicate less or nondistorted pixels.
  • The generalized deviation is the Minkowski distance of order ρ between similarity values and a measure of central tendency; this paper uses the mean.
  • Power pooling is combined with deviation pooling to control the numerical behavior and potentially improve score linearity or visualization.
  • For MDSI, the formulation includes mean-based deviation and power pooling, with the proposed parameters specified in the paper.

E. Analysis and Examples of GCS Maps

The analysis compares conventional and proposed gradient-chromaticity similarity maps across edge-distortion cases. The proposed map emphasizes major structural differences while reducing emphasis on weaker or less perceptually relevant changes.

  • The proposed d GCS map uses c GS, whereas GCS uses conventional GS, so their gradient-similarity behavior differs.
  • Removed and weak edges: For removed or weak added edges, d GCS emphasizes rope-location differences more strongly while reducing emphasis on weaker edges elsewhere.The fusion combines intermediate gradient similarities and subtracts a reference-related term to reduce emphasis on weak edges.
  • Preserved edges with different colors: For inverted edges, the proposed map is evaluated by comparing GCS1/4 with d GCS1/4, with some intermediate outputs omitted.
  • Preserved edges with different colors: The proposed gradient similarity can partially address preserved edges whose colors differ, although color similarity remains relevant because edges occupy few image pixels.

III. EXPERIMENTAL RESULTS AND DISCUSSION

The experiments evaluate MDSI across diverse natural, synthetic, and digitally retouched image-quality datasets using correlation and error metrics. Comparisons include twelve state-of-the-art IQA models.

  • Synthetic-image evaluation: The ESPL dataset contains 25 synthetic images and 500 distorted images across five distortion categories.
  • Objective evaluation uses SRC, PCC after nonlinear regression, KRC, and RMSE to assess monotonicity, linearity, rank correlation, and accuracy.
  • MDSI is compared with twelve popular or competing IQA models, including recent indices reported to outperform earlier state-of-the-art metrics.

A. Performance comparison

Across eight benchmark datasets, MDSI achieves the strongest overall performance among the compared IQA indices. It is frequently among the top-ranked models across dataset-metric combinations.

  • MDSI appears among the top indices 31 times across eight datasets, ahead of ADD-GSIM with 16 appearances.
  • Table I compares MDSI with twelve competing indices across eight benchmark datasets and highlights the top three models for each dataset and evaluation metric.
  • MDSI has the best direct and weighted overall performance across eight datasets containing 8150 images.

B. Visualization and statistical evaluation

Visualization and statistical testing examine the effect of power pooling and compare MDSI with competing metrics. The results indicate improved linearity with power pooling and strong statistical performance across datasets, with a stated F-test caveat.

  • Visualization: The reported PCC and RMSE values are computed after mapping quality scores to MOS using the logistic fitting function.
  • Visualization: Power pooling makes MDSI quality scores more linear with respect to LIVE subjective MOS, as indicated by the LPCC and PCC comparisons.
  • Statistical evaluation: The F-test may incur type I errors when IQA quality scores are not Gaussian, although the authors consider a different superiority conclusion unlikely.
  • Statistical evaluation: MDSI is significantly better than the other indices on TID2008, TID2013, ESPL, and DRIQ.
  • Statistical evaluation: MDSI obtains an overall statistical-significance score of +62 out of +72, followed by ADD-GSIM at +25.

C. Performance comparison on individual distortions

MDSI is evaluated across individual distortion types to assess reliability beyond whole-dataset averages. It achieves the strongest average correlations and relatively stable worst-case behavior among the compared indices.

  • The individual-distortion analysis identifies MDSI as a reliable index across varied distortion types.Reliability is assessed using average, minimum, and standard-deviation correlation results.
  • MDSI has the best average SRC and PCC across 61 distortion sets from six datasets.The comparison covers TID2008, CSIQ, LIVE, TID2013, VCL@FER, and ESPL.
  • MDSI, SCQI, and FSIMc perform better than the other models in the worst-case distortion results.The minimum SRC and PCC values are used as the worst-case comparison.
  • MDSI, SCQI, and FSIMc have the lowest variation across the 61 distortion-type correlation values.Lower standard deviation is treated as evidence of more consistent performance across distortion types.

D. Parameters of deviation pooling (ρ, q, o)

The deviation-pooling study examines how ρ, q, pooling choice, and map-combination parameters affect MDSI. The model uses MAD pooling and selected combination parameters while retaining robustness across a broad parameter range.

  • Parameters of deviation pooling (ρ, q, o): MDSI evaluates mean absolute deviation with ρ = 1 and standard deviation with ρ = 2 as deviation-pooling options.The study also varies the Minkowski power q and compares these settings with mean pooling.
  • Parameters of deviation pooling (ρ, q, o): Power pooling can improve linearity, increasing VSI LPCC from 0.8373 to 0.8928 and PCC from 0.9000 to 0.9011 on TID2013.These changes are reported for VSI with o = 18.
  • Parameters of deviation pooling (ρ, q, o): MDSI combines its gradient and chromaticity maps through tested summation or multiplication schemes evaluated under multiple criteria.The criteria include overall performance, individual distortions, reliability, efficiency, and simplicity.
  • Parameters of deviation pooling (ρ, q, o): The multiplication approach uses γ = 0.2 and β = 0.1 as selected parameters for combining the maps.These values were identified as good parameters after testing many parameter sets.
  • Parameters of deviation pooling (ρ, q, o): MDSI reduces its four parameters to two by setting C3 = 4C1 = 10C2, leaving C3 and α to tune.The reported experimental settings are α = 0.6, C1 = 140, C2 = 55, and C3 = 550.
  • Parameters of deviation pooling (ρ, q, o): MDSI achieves weighted average SRC above 0.90 for α ∈ [0.5, 0.7] and C3 ∈ [300, 600].This result indicates robustness across the tested parameter region.

G. Effect of chromaticity similarity maps CS and c CS

The chromaticity-map experiment compares the proposed c CS with CS within MDSI, while also examining pooling choices and computational efficiency. The proposed map generally performs better, and MDSI remains among the fastest compared models.

  • G. Effect of chromaticity similarity maps CS and c CS: On CCID2014 contrast distortions, MAD pooling and both CS and c CS are effective choices for MDSI.The experiment compares mean, MAD, and SD pooling across a range of C3 values.
  • G. Effect of chromaticity similarity maps CS and c CS: The proposed c CS performs better than CS for almost every tested pooling strategy and C3 value.The comparison is based on SRC performance within MDSI on CCID2014.
  • G. Effect of chromaticity similarity maps CS and c CS: MDSI is among the five fastest evaluated indices and is less than twice as slow as GMSD.The runtime comparison uses images of sizes 384×512 and 1080×1920 on a Core i7 3.40GHz CPU.
  • G. Effect of chromaticity similarity maps CS and c CS: MDSI is approximately 3 to 6 times faster than SCQI, 3 to 9 times faster than VSI, and 4 to 11 times faster than FSIMc.The reported speed differences vary by competing index and comparison setting.
  • G. Effect of chromaticity similarity maps CS and c CS: The paper concludes that MDSI combines effective gradient and chromaticity similarities with low complexity and real-time suitability.The conclusion covers natural and synthetic benchmark datasets.
Loading 1608.07433v4…