Source-linked AI summary
MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment
Hancheng Zhu, Leida Li, Jinjian Wu, Weisheng Dong, Guangming Shi
TL;DR
Deep NR-IQA faces limited annotated data and poor generalization from classification-oriented pre-training. MetaIQA learns shared quality priors across distortion-specific tasks with bi-level meta-learning, then fine-tunes them for unknown distortions; experiments report superior performance to state-of-the-art methods across five public databases.
Problem
Limited annotated IQA data makes deep NR-IQA models prone to overfitting, while pre-trained networks not designed for IQA generalize poorly to unknown distortions.
Method
MetaIQA uses bi-level gradient optimization across distortion-specific NR-IQA tasks to learn a shared quality prior, then fine-tunes it for an unknown-distortion task.
Results
Across five public IQA databases, MetaIQA significantly outperforms state-of-the-art NR-IQA methods in generalization ability and evaluation accuracy.
Takeaways & Limitations
The learned meta-model can adapt the shared quality knowledge from diversified distortions to unknown distortions, including authentic distortions after synthetic-distortion meta-training.
Abstract
from arXiv · showhide
Recently, increasing interest has been drawn in exploiting deep convolutional neural networks (DCNNs) for no-reference image quality assessment (NR-IQA). Despite of the notable success achieved, there is a broad consensus that training DCNNs heavily relies on massive annotated data. Unfortunately, IQA is a typical small sample problem. Therefore, most of the existing DCNN-based IQA metrics operate based on pre-trained networks. However, these pre-trained networks are not designed for IQA task, leading to generalization problem when evaluating different types of distortions. With this motivation, this paper presents a no-reference IQA metric based on deep meta-learning. The underlying idea is to learn the meta-knowledge shared by human when evaluating the quality of images with various distortions, which can then be adapted to unknown distortions easily. Specifically, we first collect a number of NR-IQA tasks for different distortions. Then meta-learning is adopted to learn the prior knowledge shared by diversified distortions. Finally, the quality prior model is fine-tuned on a target NR-IQA task for quickly obtaining the quality model. Extensive experiments demonstrate that the proposed metric outperforms the state-of-the-arts by a large margin. Furthermore, the meta-model learned from synthetic distortions can also be easily generalized to authentic distortions, which is highly desired in real-world applications of IQA metrics.
1. Introduction
NR-IQA is needed because reference images are often unavailable, but limited annotated data makes deep models prone to overfitting and poor generalization to unknown distortions. MetaIQA addresses this by learning shared quality knowledge across distortion-specific tasks for rapid adaptation.
- Motivation: Reference images are often unavailable in real-world settings, motivating NR-IQA methods that assess distorted images directly.Objective IQA can automate quality assessment for applications including image restoration, retrieval, and quality monitoring.
- Motivation: Limited annotated IQA databases make directly training deep models prone to overfitting, while classification-based pre-training remains poorly suited to unknown distortions.Human quality annotation is expensive and time-consuming, and pre-trained networks are not designed for IQA.
- Proposed approach: The proposed metric is designed to generalize across diversified distortions rather than specialize only in known distortion types.The motivation draws on humans’ ability to use quality priors from various distortions when evaluating unfamiliar images.
- Proposed approach: MetaIQA learns shared meta-knowledge among distortion-specific NR-IQA tasks through bi-level gradient optimization for fast adaptation to unknown distortions.The learned meta-knowledge serves as an ideal pre-trained model for a target NR-IQA task.
- Experiments: Experiments on five public IQA databases show significant gains over state-of-the-art NR-IQA methods in generalization ability and evaluation accuracy.The experiments include databases containing both synthetic and authentic distortions.
2. Related Work
Related work contrasts distortion-specific and general-purpose NR-IQA methods, while highlighting the data limitations of deep learning and the mismatch between classification pre-training and quality assessment. Optimization-based meta-learning offers a flexible route to learn transferable priors across distortion tasks.
- No-reference image quality assessment: Distortion-specific NR-IQA methods can align well with human perception but have limited application scope when distortion types are unknown.General-purpose methods avoid requiring the distortion type to be known in real applications.
- No-reference image quality assessment: Deep learning-based general-purpose NR-IQA methods outperform traditional methods but require abundant labeled data, whereas IQA databases are typically small.Pre-training is commonly used to mitigate overfitting caused by limited annotated IQA data.
- No-reference image quality assessment: Image-classification pre-training provides prior features, but the difference between classification and quality assessment contributes to generalization problems in deep NR-IQA models.The cited approaches initialize or fine-tune models using ImageNet or other classification-derived representations.
- Deep meta-learning: Deep meta-learning acquires prior knowledge from related tasks so models can learn new tasks from limited training data.Optimization-based methods learn initialization parameters that can be rapidly adapted through fine-tuning.
- Deep meta-learning: The paper extends optimization-based meta-learning to NR-IQA by learning shared priors from distortion-specific tasks and fine-tuning them for unknown distortions.This approach is favored because gradient optimization does not limit network structures.
3. Our Approach
MetaIQA learns a shared quality prior from multiple distortion-specific NR-IQA tasks, then fine-tunes that prior on an unknown-distortion task. Its framework uses support/query sets and bi-level optimization before producing the target quality model.
- Overall framework: MetaIQA learns a shared quality prior from multiple distortion-specific NR-IQA tasks and fine-tunes it for unknown distortions.The meta-training stage uses known distortion types; the resulting prior is adapted to a target NR-IQA task.
- Meta-training: Each distortion-specific task is divided into support and query sets for bi-level gradient optimization.Support data tentatively updates task parameters, while query data evaluates the updated model for meta-learning.
- Meta-training: The deep regression network uses convolutional layers, Global Average Pooling, and fully connected layers to predict an image quality score.For input image x, the network outputs predicted quality score y-hat, with squared Euclidean distance used against the ground-truth score.
- Meta-training: Adam updates the model on support and query sets, integrates gradients across sampled tasks, and learns the quality prior through bi-level optimization.The approach uses inner updates on task support sets and query-based updates across a mini-batch of k tasks.
- Fine-tuning for unknown distortions: Fine-tuning applies Adam to the learned quality prior on a target NR-IQA task, requiring no additional parameters before assessing unknown distortions.The resulting quality model predicts scores for query images after adaptation.
4. Experiments
Experiments evaluate MetaIQA on synthetic and authentic distortions using PLCC and SROCC, testing generalization, visualization, ablation, and parameter sensitivity. Results show strong performance across distortion settings and support the learned quality prior.
- Evaluation criteria: PLCC and SROCC evaluate agreement between predicted and ground-truth quality scores, with higher absolute values indicating better prediction performance.PLCC uses score correlation, while SROCC uses rank differences between predicted and ground-truth scores.
- Comparisons with the state-of-the-arts: 19 of 24 TID2013 distortion types and 19 of 25 KADID-10K distortion types achieve the best evaluation performance with MetaIQA.MetaIQA also exceeds 0.9 SROCC on more than half of TID2013 distortion types and leads in overall average performance on both databases.
- Comparisons with the state-of-the-arts: MetaIQA achieves the best performance on LIVE challenge and KonIQ-10K, while matching NSSADNN on CID2013 and outperforming other compared methods.The quality prior is learned from synthetic distortions and fine-tuned on authentic-distortion training sets.
- Visual analysis for quality prior model: Gradient maps locate overexposure, underexposure, motion blur, and noise in authentic LIVE challenge images after learning from synthetic distortion tasks.The visualization experiment uses four severely distorted images and presents their corresponding pixel-wise gradient maps.
- Ablation study: MetaIQA outperforms an equal-parameter direct-training baseline on all three authentic-distortion databases without changing network structure.The comparison isolates the optimization approach by contrasting meta-learning with Adam-based direct training followed by fine-tuning.
- Parameters discussion: Performance rises with k and S, drops slightly when k exceeds 5, and stabilizes when S exceeds 6; experiments therefore use k = 5 and S = 6.Increasing S from 1 to 6 produces a dramatic improvement on LIVE challenge measured by SROCC.
5. Conclusion
The paper uses meta-learning with bi-level gradient optimization to address NR-IQA generalization across distortions. Experiments show superiority over state-of-the-art methods, including adaptation from synthetic to authentic distortions.
- The proposed method learns a shared prior model from diverse NR-IQA tasks and fine-tunes it for unknown distortions.Bi-level gradient optimization captures shared meta-knowledge before target-task adaptation.
- The approach outperforms state-of-the-art NR-IQA methods in both generalization ability and evaluation accuracy.The evidence comes from experiments on five public IQA databases.
- A quality prior model learned from synthetic distortions can be quickly adapted to authentically distorted images.The paper identifies this transfer as relevant to real-world quality-assessment model design.