PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning a No-Reference Quality Assessment Model of Enhanced Images With Big Data

Ke Gu, Dacheng Tao, Junfei Qiao, Weisi Lin

arXiv:1904.08632v1cs.CV

TL;DR

The paper asks how to assess enhanced-image quality when original references are unavailable and how to use that assessment for enhancement. It proposes BIQME, a 17-feature blind metric learned with large training data, and applies it to quality-optimized enhancement. Experiments on nine datasets report superior performance against typical IQA methods and successful enhancement across several image types.

  • Problem

    Enhanced images can improve or degrade visual quality, while original references are often unavailable for assessing them.

  • Method

    BIQME extracts 17 features from five enhancement-related factors and uses a regression module learned from more than 100,000 images, then guides successive brightness and contrast correction.

  • Results

    Experiments on nine datasets report that BIQME outperforms same-type NR-IQA methods and general-purpose FR- and RR-IQA methods, with especially encouraging performance for contrast-changed images and enhanced haze images.

  • Takeaways & Limitations

    The framework supports blind quality prediction and quality-guided enhancement of natural, low-contrast, low-light and dehazed images.

Abstract

from arXiv · show

In this paper we investigate into the problem of image quality assessment (IQA) and enhancement via machine learning. This issue has long attracted a wide range of attention in computational intelligence and image processing communities, since, for many practical applications, e.g. object detection and recognition, raw images are usually needed to be appropriately enhanced to raise the visual quality (e.g. visibility and contrast). In fact, proper enhancement can noticeably improve the quality of input images, even better than originally captured images which are generally thought to be of the best quality. In this work, we present two most important contributions. The first contribution is to develop a new no-reference (NR) IQA model. Given an image, our quality measure first extracts 17 features through analysis of contrast, sharpness, brightness and more, and then yields a measre of visual quality using a regression module, which is learned with big-data training samples that are much bigger than the size of relevant image datasets. Results of experiments on nine datasets validate the superiority and efficiency of our blind metric compared with typical state-of-the-art full-, reduced- and no-reference IQA methods. The second contribution is that a robust image enhancement framework is established based on quality optimization. For an input image, by the guidance of the proposed NR-IQA measure, we conduct histogram modification to successively rectify image brightness and contrast to a proper level. Thorough tests demonstrate that our framework can well enhance natural images, low-contrast images, low-light images and dehazed images. The source code will be released at https://sites.google.com/site/guke198701/publications.

I. INTRODUCTION

The paper addresses the need to assess enhanced images without original references, since enhancement can improve or degrade visual quality. It introduces BIQME, a blind metric using enhancement-related features and big-data learning, alongside a quality-optimization framework.

  • Motivation: Objective IQA methods are useful in practical systems because subjective assessment is costly and time-consuming.Objective metrics can support applications such as enhancement and tone-mapping.
  • Motivation: Enhanced images may improve visual quality beyond originally captured images, while improper enhancement can degrade it.This motivates dedicated IQA for enhancement rather than focusing only on conventional artifacts.
  • Research Gap: Existing full- and reduced-reference measures cannot operate when original references are unavailable, creating a need for blind methods.A prior blind metric also assumes natural images have the highest quality and overlooks important enhancement influences.
  • Contributions: BIQME extracts 17 features from contrast, sharpness, brightness, colorfulness and naturalness, then fuses them with a learned regression module.The framework is designed specifically for blind quality assessment of enhanced images.
  • Contributions: The paper trains BIQME with 100,000 samples, reports stronger performance than many FR, RR and NR methods, and explores BIQME-based enhancement optimization.Its contributions include an opinion-unaware blind metric and a robust enhancement technology.

II. NO-REFERENCE QUALITY METRIC

BIQME represents enhanced-image quality through 17 features covering five influencing factors, then uses a regression module learned from extensive training data to infer one quality score.

  • Metric Design: BIQME extracts 17 features representing contrast, sharpness, brightness, colorfulness and naturalness, then fuses them through learned regression.The regression module converts the heterogeneous feature set into the final visual-quality score.

A. Feature Extraction

The feature-extraction stage measures enhancement-related properties across contrast, sharpness, brightness, colorfulness and naturalness. It combines global and local perceptual cues with natural-scene statistics to characterize enhanced-image quality.

  • Contrast: Contrast features combine phase-congruence entropy with contrast energy from grayscale and opponent-color channels.The resulting set is Fct = {Epc, CEgr, CEyb, CErg}.
  • Contrast: The phase-congruence entropy emphasizes pixels corresponding to the 40% largest values in the detected phase-congruence map.Phase congruence is used to identify perceptually informative image features.
  • Sharpness: Sharpness is estimated from log-energy in wavelet subbands after three-level 9/7 discrete wavelet decomposition.Only the second and third levels are retained because they contain more sharp details; HH subbands receive larger weights.
  • Brightness: Brightness features characterize signal levels using six measurements, while the implementation temporarily targets 8-bit images in the range [0,255].The brightness feature set is Fb = {Em1, Em2, Em3, Em4, Em5, Em6}.
  • Colorfulness: Colorfulness features use a practical global colorfulness metric with a significance-rectifying parameter whose optimal value is κ = 0.3.The resulting feature set is Fcl = {S, C}.
  • Naturalness: Naturalness features use natural-scene statistics after local mean removal and divisive normalization, with naturalness primarily penalizing over-enhancement.The normalized coefficients are modeled with a zero-mean generalized Gaussian distribution.

B. Quality Prediction

The BIQME quality-prediction stage integrates 17 enhancement-related features through SVR regression trained on over 100,000 generated images labeled with C-PCQI scores.

  • Regression module: 17 enhancement-related features are converted into one quality score because direct linear weighting would require many difficult-to-estimate parameters.The paper considers dimensionality reduction but finds it difficult because features have different roles and dimensions.
  • Training data: Over 100,000 images are used to train the BIQME regression module, addressing overfitting concerns associated with smaller training sets.The training set is larger than the 655-image CCID2014 database.
  • Training labels: C-PCQI scores replace human opinion ratings as labels for the 100,162 training images.C-PCQI extends PCQI by incorporating colorfulness, while its components measure mean intensity, contrast change, structural distortion, and color saturation.
  • Regression module: Support vector regression with an RBF kernel learns the regression module from the training samples.The training process determines the regression parameters associated with the RBF kernel.
  • Learning strategy: The proposed strategy is described as data-fitting adaptation: a high-performance FR-IQA model supplies approximate ground-truth scores for learning an NR-IQA model.This is contrasted with model distillation, which transfers knowledge from a cumbersome trained model to a smaller model.

III. EXPERIMENTAL RESULTS AND DISCUSSIONS

The experimental-results section evaluates BIQME against up to 16 state-of-the-art IQA approaches across nine enhancement-related databases.

  • Evaluation scope: BIQME is evaluated and compared with up to 16 state-of-the-art IQA approaches on nine enhancement-related databases.The evaluation concerns the performance of the proposed blind metric.

A. Experimental Setup

The experimental setup compares IQA methods across enhancement-related databases using correlation criteria and visual performance plots.

  • Compared methods: The benchmark includes full-reference methods FSIM, LTG, VSI, and PSIM; reduced-reference methods RRED and FTQM; and blind IQA methods including BRISQUE, NFERM, and FANG.These methods represent the principal IQA categories used for comparison.
  • Testing datasets: Nine relevant subjective databases include CID2013 and CCID2014 plus contrast-enhancement subsets from TID2008, CSIQ, TID2013, and SIQAD.CID2013 and CCID2014 were constructed specifically for image-quality evaluation of contrast changes.
  • Evaluation criteria: Performance is assessed with Pearson linear correlation, Spearman rank correlation, and Kendall rank-order correlation coefficients.Values approaching one indicate superior agreement with human opinion ratings for all three criteria.
  • Figure 2: Figure 2 compares BIQME, FANG, NFERM, and BRISQUE across six datasets using PLC, SRC, and KRC bars.Blue, red, and green bars represent PLC, SRC, and KRC, respectively.
  • Figure 3: Figure 3 presents scatter plots for BIQME and FANG under leave-one-out cross-validation on six datasets.The leave-one-out procedure trains on images associated with the remaining original images before predicting the held-out group.

B. Performance Results

The proposed BIQME metric is evaluated through feature-effectiveness tests, cross-dataset comparisons, scatter-plot analysis, and runtime measurement. Across these evaluations, it shows reliable predictions, broad comparative superiority, and subsecond assessment on tested images.

  • Feature Effectiveness: BIQME attained encouraging performance, especially for contrast-changed images and enhanced haze images, in repeated train-test experiments.The evaluation used random 80%-train/20%-test splits repeated 1,000 times for six datasets and SRC results for three additional subsets.
  • Feature Effectiveness: BIQME generated more reliable quality predictions than FANG across CID2013, CCID2014, TID2008, CSIQ, TID2013, and SIQAD.Its scatter-plot sample points were reported as consistently and substantially closer to the black diagonal lines indicating perfect performance.
  • Performance Comparison: BIQME outclassed general-purpose FR-, RR-, and NR-IQA methods on average across the reported testing datasets.The paper reports both direct-mean and image-count-weighted mean performance indices.
  • Runtime: BIQME assessed a 768 × 576 image in less than one second despite computing multiple feature types.Runtime was measured on the CCID2014 database using MATLAB2015 on a 3.20GHz CPU and 16GB-memory desktop.

IV. QUALITY-BASED IMAGE ENHANCEMENT

The BOIEM framework uses BIQME-guided optimization to successively rectify image brightness and contrast, targeting enhancement across multiple image types. It combines modified AGCWD and RICE with blind quality assessment to select processing parameters.

  • Framework: BOIEM combines modified AGCWD and RICE to successively rectify image brightness and contrast.The cascade uses parameters λb, λs and λe selected through BIQME-based optimization.
  • Framework: AGCWD weights the image PDF before computing the CDF, with λb controlling the weighting strength.The parameter λb is applied to the PDF range before the weighted PDF is used to produce the enhanced image.
  • Parameter optimization: A fixed λb can cause AGCWD to over-enhance images, producing excessive brightness, particularly in naturally well-lit images.This motivates adaptive parameter selection rather than empirical assignment of a constant weight.
  • Parameter optimization: RICE optimizes histogram modification using uniform, original, and positively skewed histogram targets with weighting parameters λe and λs.Its formulation assumes properly enhanced images have a near-uniform histogram, remain close to the original, and exhibit positively skewed statistics.
  • Parameter optimization: BOIEM performs six BIQME evaluations to select brightness and contrast-related parameter candidates, accelerating quality-guided optimization.The first three evaluations enumerate λb candidates {0.3, 0.5, 0.7}; the remaining evaluations select λs and λe candidates.
  • Results: BOIEM enhances natural, low-contrast, low-light, and dehazed images with suitable brightness and contrast while displaying more details.The comparison includes original images, AGCWD, RICE, and BOIEM; AGCWD can over-brighten, while RICE is ineffective for low-light images.

V. CONCLUSION

The paper presents BIQME as a blind quality framework using 17 features and applies it to quality-guided enhancement. Its framework covers four image categories, while the authors identify broader feature and enhancement extensions as future work.

  • Contributions: BIQME predicts enhanced-image quality blindly from 17 features covering contrast, sharpness, brightness, colorfulness, and naturalness.Experiments on nine relevant image datasets found the metric superior to same-type NR-IQA methods.
  • Implications: Feature types are independent, allowing parallel computation, and the IQA framework can incorporate additional features.These properties are presented as opportunities to improve computational efficiency and performance.
  • Contributions: The framework applies BIQME optimization to successive brightness and contrast rectification for natural, low-contrast, low-light, and dehazed images.This connects quality assessment with robust enhancement rather than limiting IQA to quality-score prediction.
  • Future work: Future work includes saliency-based feature modification and extending blind IQA to denoising, deblurring, and super-resolution.The authors also note that adding procedures such as haze removal could make enhancement more universal.
Loading 1904.08632v1…