Source-linked AI summary
A Haar Wavelet-Based Perceptual Similarity Index for Image Quality Assessment
Rafael Reisenhofer, Sebastian Bosse, Gitta Kutyniok, Thomas Wiegand
TL;DR
Perceptual image quality assessment is needed for distorted signals, but psychometric testing is too costly and slow for many practical uses. The paper introduces HaarPSI, an inexpensive full-reference measure using Haar wavelet coefficients for local similarity and spatial weighting. Across benchmark databases, it shows strong agreement with human scores, generally outperforming established metrics while remaining computationally efficient.
Problem
Psychometric quality tests are the gold standard for perceptual assessment but are expensive and time-consuming, limiting their feasibility for real-time transmission-system optimization and monitoring.
Method
HaarPSI uses coefficients from three stages of a discrete Haar wavelet transform, with high-frequency information defining local similarity and lower-frequency information weighting spatial importance.
Results
HaarPSI achieves significantly higher or statistically indistinguishable Spearman correlations than nine popular state-of-the-art metrics on grayscale conversions, while ranking below VSI for color images on TID 2013.
Takeaways & Limitations
Its simple computational structure and comparatively short execution time suggest high applicability for practical image-quality assessment tasks.
Takeaways & Limitations
HaarPSI can underperform on Gaussian-blur distortions because it relies almost exclusively on high-frequency information and may be too sensitive to low-pass filtering.
Abstract
from arXiv · showhide
In most practical situations, the compression or transmission of images and videos creates distortions that will eventually be perceived by a human observer. Vice versa, image and video restoration techniques, such as inpainting or denoising, aim to enhance the quality of experience of human viewers. Correctly assessing the similarity between an image and an undistorted reference image as subjectively experienced by a human viewer can thus lead to significant improvements in any transmission, compression, or restoration system. This paper introduces the Haar wavelet-based perceptual similarity index (HaarPSI), a novel and computationally inexpensive similarity measure for full reference image quality assessment. The HaarPSI utilizes the coefficients obtained from a Haar wavelet decomposition to assess local similarities between two images, as well as the relative importance of image areas. The consistency of the HaarPSI with the human quality of experience was validated on four large benchmark databases containing thousands of differently distorted images. On these databases, the HaarPSI achieves higher correlations with human opinion scores than state-of-the-art full reference similarity measures like the structural similarity index (SSIM), the feature similarity index (FSIM), and the visual saliency-based index (VSI). Along with the simple computational structure and the short execution time, these experimental results suggest a high applicability of the HaarPSI in real world tasks.
1 Introduction
Human-perceived image quality is important for evaluating compressed or transmitted signals, but psychometric testing is costly and impractical for real-time systems. HaarPSI addresses this need with an inexpensive full-reference measure built from Haar wavelet features and validated against human opinion scores.
- Motivation: Psychometric tests provide the gold standard for perceptual quality assessment, but their expense and time requirements prevent routine real-time use.This limitation motivates computational image quality assessment methods.
- IQA approaches: Full-reference, reduced-reference, and no-reference IQA methods differ in their reliance on reference-image information and application scope.Full-reference methods require a reference image, reduced-reference methods use extracted reference features, and no-reference methods use only the distorted image.
- Contribution: HaarPSI is a novel, computationally inexpensive full-reference measure that uses high-frequency Haar coefficients for local similarity and low-frequency coefficients for spatial weighting.The measure is designed to assess both local correspondence and the relative importance of image locations.
- Contribution: Six two-dimensional Haar filters capture horizontal and vertical edges across frequency scales, implementing orientation and spatial-frequency selectivity with a simple model.The paper investigates whether these elementary visual-system properties suffice for state-of-the-art correlation with human opinion scores.
- Relation to prior work: HaarPSI simplifies FSIM by replacing 16 complex-valued filters and adaptive thresholding with six discrete Haar filters and separate frequency roles for similarities and weights.The paper reports that these simplifications reduce computational effort while producing consistently higher correlations with human mean opinion scores.
- Evaluation: Across four large databases, HaarPSI generally correlates more strongly with human opinion scores than state-of-the-art full-reference metrics and executes faster than VSI and FSIM.It ranks below VSI in only one reported test case.
2 The Haar Wavelet-Based Perceptual Similarity Index
HaarPSI builds perceptual similarity from a compact Haar-wavelet filterbank: high-frequency responses produce local similarities, while low-frequency responses weight their spatial importance. The method extends to color images and uses two optimized parameters, C and α.
- Haar filterbank: HaarPSI uses six two-dimensional Haar filters, whose responses capture horizontal and vertical structures across frequency scales.The filters are constructed from one-dimensional Haar filters using outer products and dyadic upsampling.
- Relation to FSIM: Unlike FSIM, HaarPSI separates local similarities and weights and computes both from only six discrete Haar filters rather than 16 complex-valued filters with additional processing.This design is presented as a computationally simpler analogue of FSIM.
- Nonlinear mapping: A logistic nonlinearity is applied to local similarities to model thresholding and improve the relationship between HaarPSI values and human opinion scores.The subsequent squaring operation spreads values in the unit interval and aims to make individual scores more interpretable.
- Similarity and weighting: High-frequency Haar responses define local similarity maps through the similarity function S, while a low-frequency response supplies location-dependent weights.The resulting similarity is formed as a weighted average of local similarities.
- Color extension: HaarPSI extends to color images in YIQ by adding a chroma-based similarity map for the I and Q channels.The luminance and chroma components use corresponding local similarity and weight maps.
- Parameter selection: C and α are optimized on quarter-sized random subsets of TID 2008, TID 2013, LIVE, and CSIQ to maximize the mean of four SROCC values.The optimization uses database MOS values and proceeds in two steps, beginning with a grid search.
3 Experimental Results
The evaluation compares HaarPSI with image-quality metrics across four diverse, quality-annotated databases and examines both overall and distortion-specific agreement with human scores. It also considers color-versus-grayscale inputs, statistical significance, execution time, and the HaarPSI–DMOS relationship.
- Evaluation setup: Four large benchmark databases provide quality-annotated images spanning varied references, distortions, observers, viewing conditions, and presentation procedures.The databases include LIVE, TID 2008, CSIQ, and the cited four-database evaluation set; their designs differ in distortion coverage and subjective-rating protocols.
- Evaluation setup: SROCC is used throughout to compare monotonic relationships between computational quality measures and human mean opinion scores.Statistical significance is assessed using ITU guidelines, with differences highlighted when the null hypothesis is rejected at p < 0.05.
- Evaluation setup: The experiments test ten similarity measures on both original color images and grayscale conversions, reflecting differences in their input designs.Metrics defined only for grayscale images may take slightly longer on color images because of the additional conversion step.
- Overall results: HaarPSI is computed faster than VSI and FSIM, the metrics with the second- and third-highest correlations with human opinion scores.Mean execution time is reported alongside average SROCC across the four databases in Table 2.
- Additional analyses: Distortion-specific correlations and HaarPSI–DMOS scatter plots provide complementary views beyond aggregate correlation coefficients.Figure 4 covers LIVE, TID 2008, TID 2013, and CSIQ, while the distortion-specific analysis addresses applications such as compression and denoising.
- Overall results: HaarPSI achieves higher correlations with human opinion scores than the other considered full-reference metrics in all but one test case, where VSI ranks first.The comparison includes state-of-the-art measures such as SSIM, FSIM, and VSI; table annotations identify statistically significant differences at p < 0.05.
4 Conclusion
HaarPSI is a fast, wavelet-based full-reference quality measure that generally tracks human opinion well, though its performance depends on distortion type and benchmark scope. Its simplified design improves interpretability and execution time but leaves several perceptual mechanisms and performance differences unexplained.
- Evaluation: More than 5000 distorted images across four databases supported HaarPSI’s validity, with significantly higher or statistically indistinguishable grayscale SROCCs than nine state-of-the-art metrics.For color images, HaarPSI ranked second to VSI on TID2013.
- Evaluation: HaarPSI generally led complete-database evaluations, but Gaussian-blur subsets were the consistent case where its performance was lower than competing metrics.This distinction reconciles apparently contradictory full-database and distortion-specific results.
- Limitations: HaarPSI’s reliance on high-frequency information may make it overly sensitive to distortions based purely on low-pass filtering.The authors identify this as an explanation for weaker performance on Gaussian blur.
- Parameterization: The constants C and α can be retuned for individual distortion types, whereas their original values optimize overall performance across databases.Increasing C reduces sensitivity to high-frequency changes relative to lower-frequency weights; larger α makes severe changes dominate.
- Limitations: The benchmark evidence may overstate generality because it uses limited reference images and distortion types, while HaarPSI models orientation selectivity only through horizontal and vertical filters.Divisive normalization is also absent, leaving its potential benefit an open question.
- Wavelet choice: Haar filters were both computationally efficient and qualitatively strongest among the tested alternative wavelets.The measure can nevertheless be defined using other wavelet families.
A Pearson Product-Moment Correlations
Table 5 reports Pearson correlations between image quality assessment metrics and human mean opinion scores. The comparisons mark whether correlations are lower or higher than HaarPSI, with statistical significance noted.
- Table 5 compares IQA metrics using Pearson correlations with human mean opinion scores.
- One compared metric has a lower correlation than HaarPSI, with the difference statistically significant at p < 0.05.
- Another compared metric has a higher correlation than HaarPSI, with the difference statistically significant at p < 0.05.
- The highest correlation in each row is boldfaced, and all correlations were obtained without nonlinear regression.