PaperlayerView exact revision on arXiv

Source-linked AI summary

Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment

Dingquan Li, Tingting Jiang, Ming Jiang

arXiv:2008.03889v1eess.IVcs.CVcs.MM

TL;DR

IQA models commonly train with MAE or MSE, which can converge slowly. This paper introduces Norm-in-Norm, which compares separately normalized predictions and subjective scores and fits a linear mapping after training. On KonIQ-10k and CLIVE, it reports faster convergence and better performance than MAE or MSE, with state-of-the-art performance on in-the-wild IQA.

  • Problem

    Most IQA models use MAE or MSE and exhibit slow convergence, making faster training desirable as dataset sizes grow.

    Source: §1 · p. 1

  • Method

    Norm-in-Norm separately normalizes predicted and subjective quality scores, computes a norm of their differences, and uses least squares after training to map predictions to subjective scores.

    Source: §1 · p. 1

  • Results

    On KonIQ-10k and CLIVE, the proposed loss yields faster convergence and better performance than MAE or MSE, and the model outperforms state-of-the-art models on in-the-wild IQA.

    Source: §1 · p. 3, §5 · p. 8

  • Takeaways & Limitations

    Faster convergence offers a way to reduce training time, a concern as IQA datasets grow.

    Source: §1 · p. 1, §5 · p. 8

    Optimally choosing the loss hyperparameters p and q for a specific regression task remains a future direction.

    Source: §5 · p. 9

Abstract

from arXiv · show

Currently, most image quality assessment (IQA) models are supervised by the MAE or MSE loss with empirically slow convergence. It is well-known that normalization can facilitate fast convergence. Therefore, we explore normalization in the design of loss functions for IQA. Specifically, we first normalize the predicted quality scores and the corresponding subjective quality scores. Then, the loss is defined based on the norm of the differences between these normalized values. The resulting "Norm-in-Norm'' loss encourages the IQA model to make linear predictions with respect to subjective quality scores. After training, the least squares regression is applied to determine the linear mapping from the predicted quality to the subjective quality. It is shown that the new loss is closely connected with two common IQA performance criteria (PLCC and RMSE). Through theoretical analysis, it is proved that the embedded normalization makes the gradients of the loss function more stable and more predictable, which is conducive to the faster convergence of the IQA model. Furthermore, to experimentally verify the effectiveness of the proposed loss, it is applied to solve a challenging problem: quality assessment of in-the-wild images. Experiments on two relevant datasets (KonIQ-10k and CLIVE) show that, compared to MAE or MSE loss, the new loss enables the IQA model to converge about 10 times faster and the final model achieves better performance. The proposed model also achieves state-of-the-art prediction performance on this challenging problem. For reproducible scientific research, our code is publicly available at https://github.com/lidq92/LinearityIQA.

1 INTRODUCTION

IQA models commonly use MAE or MSE, but these losses can converge slowly. The paper proposes a normalization-based loss for IQA, analyzes its properties and metric connections, and evaluates it on in-the-wild images.

1 INTRODUCTION

  • Deep IQA models commonly use MAE or MSE and can take more than a day to converge on a dataset of about 10,000 images.The paper argues that faster convergence is preferable as training datasets grow.Source: §1 · p. 1
  • Inspired by normalization methods, the paper explores normalizing network predictions within IQA loss functions, although output normalization is rarely recommended.The motivation also cites prior work where normalizing network outputs accelerated convergence for image super-resolution.Source: §1 · p. 1
  • Norm-in-Norm separately normalizes predicted and subjective quality scores, compares them through a norm of their differences, and fits their linear mapping with least squares after training.The loss encourages linear predictions with respect to subjective quality scores; the learned mapping is used to produce final test predictions.Source: §1 · p. 1
  • The proposed loss includes a PLCC-induced loss as a special case, while a variant is connected to RMSE.PLCC and RMSE are IQA performance criteria.Source: §1 · pp. 1–2
  • Theoretical analysis attributes more stable, predictable gradients and a smoother loss landscape to embedded normalization, which is conducive to faster convergence.The analysis links this behavior to stronger Lipschitzness and β-smoothness.Source: §1 · p. 2
  • On KonIQ-10k, Norm-in-Norm reaches the grey-dash-line prediction performance after one training epoch, about 10 times faster than MAE and MSE, and yields better prediction performance than both losses.Source: §1 · p. 2, Figure 1 · p. 2
  • For in-the-wild IQA, the no-reference model aggregates and fuses multilevel deep features to address content dependence and distortion complexity.The paper identifies those two properties as challenges in assessing in-the-wild images.Source: §1 · p. 2
  • The proposed model also outperforms state-of-the-art models on the challenging in-the-wild IQA problem.The evaluation addresses in-the-wild image quality, where content dependency and distortion complexity are identified as challenges.Source: §1 · pp. 2–3

2 “NORM-IN-NORM” LOSS

The Norm-in-Norm loss normalizes predicted and subjective quality scores separately, then penalizes differences between them to encourage linear predictions. Special cases connect the loss to PLCC, while a variant connects it to RMSE after linear mapping.

2.1 Loss Computation

  • The loss computes score statistics, normalizes predictions and subjective scores by their own means and centered norms, then measures the norm of their differences.It encourages predictions that are linearly correlated with subjective quality scores.Source: §2.1 · pp. 3–4

2.2 A Special Case: PLCC-induced Loss

  • When p=q=2, the Norm-in-Norm loss is equivalent to the PLCC-induced loss.The paper identifies this special case with the loss based on Pearson’s linear correlation coefficient.Source: §2.2 · p. 4

2.3 A Variant and Its Connection to RMSE

  • The loss variant focuses on linear predictions and requires normalized predicted and subjective scores to be linearly correlated.The paper introduces least squares regression to find a linear mapping between predicted scores and subjective scores.Source: §2.3 · p. 4
  • With p=q=2, a special case of the variant is connected to RMSE between linearly mapped predictions and subjective scores.The stated parameter choice is p=q=2.Source: §2.3 · p. 4

3 THEORETICAL ANALYSIS

For q = 2, embedded normalization improves the proposed loss’s Lipschitzness and β-smoothness, reducing gradient magnitude and the Hessian quadratic form. The resulting gradients are more stable and predictable, yielding a smoother loss landscape and faster, more robust IQA training.

3 THEORETICAL ANALYSIS

  • Lipschitzness is indicated by gradient magnitude, while β-smoothness is indicated by the Hessian’s quadratic form in the gradient direction.The paper defines β-smoothness as Lipschitzness of the gradient.Source: §3 · p. 4
  • For q = 2, embedded normalization reduces gradient magnitude and the Hessian quadratic form, improving Lipschitzness and β-smoothness and supporting more stable, predictable gradients.The paper links the smoother loss landscape to more robust training and faster convergence.Source: §3 · p. 4
  • Lipschitzness improves when the gradient sum deviates from zero or correlates with normalized predictions; a coefficient greater than 1 also contributes.The paper states that the coefficient is greater than 1 in practice.Source: §3 · p. 5
  • β-smoothness improves when normalized labels and predictions, or the gradient and normalized predictions, are not linearly correlated; a coefficient greater than 1 also contributes.The paper identifies these relationships as conditions that improve β-smoothness.Source: §3 · p. 5

4 VERIFICATION ON QUALITY ASSESSMENT OF IN-THE-WILD IMAGES

This section evaluates Norm-in-Norm for in-the-wild IQA against MAE and MSE on KonIQ-10k and CLIVE, using a multi-level feature model. The experiments report faster convergence, stronger prediction performance, and greater stability across backbones and training settings.

4 VERIFICATION ON QUALITY ASSESSMENT OF IN-THE-WILD IMAGES

  • The experiments test whether Norm-in-Norm improves convergence and prediction performance for the challenging task of in-the-wild image quality assessment.The stated challenge is handling image-content dependency and distortion complexity.Source: §4 · p. 5

4.1 IQA Framework

  • The framework extracts and fuses multi-level deep features to address content dependency and distortion complexity, then applies least squares regression to map predictions to subjective scores.The learned mapping is applied to test predictions.Source: §4.1 · p. 5

4.2 Experimental Setup

  • Experiments use KonIQ-10k for training, validation, and testing, and CLIVE for cross-dataset evaluation; the default setup uses p = 1, q = 2 and batch size 8.The reported test metrics are SROCC, PLCC, and RMSE on KonIQ-10k; CLIVE evaluation uses SROCC and PLCC as reported in the results.Source: §4.2 · p. 6

4.3.1 Model Convergence With MAE, MSE, or the Proposed Loss.

  • About 10 times slower: MAE and MSE take roughly ten times longer than Norm-in-Norm to reach the indicated prediction performance on KonIQ-10k.The final convergence state also has higher SROCC/PLCC and lower RMSE than MAE and MSE.Source: §4.3.1 · p. 6
  • For KonIQ-10k, p = 1, q = 2 is the best tested choice, while p = 1 is generally better than p = 2.The authors note that p = 2 is more sensitive to outliers and that the best p,q may depend on the task and dataset.Source: §4.3.1 · pp. 6–7, Table 1 · p. 6

4.3.3 Training Stability With MAE, MSE, or the Proposed Loss Un-

  • Norm-in-Norm is more stable than MAE and MSE across learning rates and batch sizes, and when the fine-tuned rate is large.Its validation performance is better with batch size 16 than 8, whereas MAE and MSE do not improve; the authors use batch size 8 in other experiments for fair comparison.Source: §4.3.3 · p. 7

4.3.4 Performance Consistency for MAE, MSE, or the Proposed Loss

  • Across ResNet-18, ResNet-34, ResNet-50, and ResNeXt-101, Norm-in-Norm consistently achieves the best reported prediction performance.The comparison reports PLCC on KonIQ-10k test and SROCC on CLIVE; its ResNeXt-101 scatter points are more centered on the diagonal than the alternatives.Source: §4.3.4 · p. 8, Figure 6 · p. 8, Figure 7 · p. 8

4.4 Performance Comparison with SOTA

  • SROCC values of 0.938 on KonIQ-10k test and 0.836 on CLIVE show that the final model can outperform the compared state-of-the-art models.The comparison includes classic and deep learning-based IQA models; the reported values are for the model combining the loss and its variant with weights 1 and 0.1.Source: Table 2 · p. 8, §4.4 · p. 8

5 CONCLUSION

The paper proposes normalized loss functions for IQA, connecting them to PLCC and RMSE and reporting faster convergence and better performance than MAE and MSE on KonIQ-10k and CLIVE. The loss is invariant to subjective-rating scale, while choosing p and q optimally for a task remains future work.

5 CONCLUSION

  • On KonIQ-10k and CLIVE, the proposed loss yields faster convergence and better performance than MAE and MSE, and its normalization improves loss-landscape smoothness.The paper also reports state-of-the-art prediction performance for in-the-wild image quality assessment in the abstract.Source: §5 · p. 8
  • The loss is invariant to subjective-rating scale, allowing datasets with different rating scales to be mixed when training a universal IQA model.Source: §5 · p. 8
  • How to choose p and q optimally for a specific regression task remains a future research direction.Source: §5 · p. 9

A DERIVATION OF c IN EQN. (6)

This appendix derives the constant c in Equation 6 using a norm inequality and Minkowski’s inequality. The derivation applies the bound separately according to whether p is below or at least q.

A DERIVATION OF c IN EQN. (6)

B ˆb CURVE IN OUR EXPERIMENT

The appendix examines the estimated b-hat curves across iterations and epochs and reports that b-hat is larger than 1 in practice.

B ˆb CURVE IN OUR EXPERIMENT

C PROOF OF THEOREM 3.1 (LIPSCHITZNESS)

The appendix fragment uses a centering condition on the normalized vector when transforming Equation 19, and identifies Figure 9 as an average b-hat curve over epochs.

C PROOF OF THEOREM 3.1 (LIPSCHITZNESS)

D PROOF OF THEOREM 3.2 (β-SMOOTHNESS)

The proof attributes a reduction in the loss-Hessian quadratic form to embedded normalization. With SGD or Adadelta, the proposed loss outperforms MAE and MSE when each uses its best initial learning rate.

D PROOF OF THEOREM 3.2 (β-SMOOTHNESS)

E.1 Different Optimizers

  • With SGD or Adadelta, the proposed loss outperforms MAE and MSE when each uses its best initial learning rate.Table 3 reports PLCC comparisons under these optimizers.Source: §E.1 · p. 11, Table 3 · p. 11
  • The best initial learning rates are 1e-1 for the proposed loss, 1e-2 for MAE, and 1e-4 for MSE.Source: §E.1 · p. 11
  • Model performance with SGD or Adadelta is very sensitive to the initial learning rate.Source: §E.1 · p. 11

E.2 Different Architectures

The proposed loss remains superior to the comparison losses when AlexNet or VGG-16 is used without batch-normalization layers.

E.2 Different Architectures

  • With AlexNet or VGG-16 backbones, the proposed loss is better than MSE; its superiority is also reported with ResNet-based backbones.The authors conclude that batch-normalization layers do not affect the new loss's superiority.Source: §E.2 · p. 11

E.3 t-test

The t-test uses 10 random CLIVE splits to assess performance comparisons. For PLCC, p = 1, q = 2 significantly outperforms p = 2, q = 1 and p = 2, q = 2, but is on par with p = 1, q = 1.

E.3 t-test

  • The Table 1 and Table 2 results used one owner-provided KonIQ-10k split; significance tests used 10 random CLIVE splits with ResNet-18.Source: §E.3 · pp. 11–12
  • For PLCC, p = 1, q = 2 significantly outperforms p = 2, q = 1 (p-value 0.018) and p = 2, q = 2 (p-value 0.028).Source: §E.3 · p. 12
  • For PLCC, p = 1, q = 2 is on par with p = 1, q = 1 (p-value 0.385).Source: §E.3 · p. 12
  • The combined loss l + 0.1l′ is slightly, but not significantly, better than l (p-value 0.172).Source: §E.3 · p. 12

Related papers

Loading 2008.03889v1…