Source-linked AI summary
ARNIQA: Learning Distortion Manifold for Image Quality Assessment
Lorenzo Agnolucci, Leonardo Galteri, Marco Bertini, Alberto Del Bimbo
TL;DR
NR-IQA seeks to measure image quality when no high-quality reference is available, while supervised approaches rely on expensive ratings and prior self-supervised methods can produce content-dependent embeddings. ARNIQA learns a distortion manifold by matching crops from different images degraded alike and synthesizing varied distortion sequences. It reports state-of-the-art performance across synthetic and authentic distortions, alongside data efficiency, generalization, and robustness gains.
Problem
Supervised NR-IQA depends on costly quality ratings, while existing self-supervised methods can learn content-dependent embeddings from crops of the same distorted image.
Source: §1 · pp. 1–2Method
ARNIQA synthetically degrades images with ordered distortion sequences and trains a self-supervised encoder to match crops from different images degraded alike.
Source: §5 · pp. 8–9Results
Takeaways & Limitations
The learned distortion-manifold representations can be mapped to quality scores with a simple linear regressor, without fine-tuning encoder weights.
Source: §5 · p. 9The training focus is on degradation compositions that alter image content without making it unrecognizable, despite also aiming to learn regions for very severe degradations.
Source: §3.1 · p. 4
Abstract
from arXiv · showhide
No-Reference Image Quality Assessment (NR-IQA) aims to develop methods to measure image quality in alignment with human perception without the need for a high-quality reference image. In this work, we propose a self-supervised approach named ARNIQA (leArning distoRtion maNifold for Image Quality Assessment) for modeling the image distortion manifold to obtain quality representations in an intrinsic manner. First, we introduce an image degradation model that randomly composes ordered sequences of consecutively applied distortions. In this way, we can synthetically degrade images with a large variety of degradation patterns. Second, we propose to train our model by maximizing the similarity between the representations of patches of different images distorted equally, despite varying content. Therefore, images degraded in the same manner correspond to neighboring positions within the distortion manifold. Finally, we map the image representations to the quality scores with a simple linear regressor, thus without fine-tuning the encoder weights. The experiments show that our approach achieves state-of-the-art performance on several datasets. In addition, ARNIQA demonstrates improved data efficiency, generalization capabilities, and robustness compared to competing methods. The code and the model are publicly available at https://github.com/miccunifi/ARNIQA.
1. Introduction
NR-IQA evaluates image quality without a high-quality reference, but supervised methods depend on costly ratings and prior self-supervised methods can learn content-dependent representations. ARNIQA learns a distortion manifold by matching differently sourced images degraded alike, using varied synthetic distortions, and reports strong performance and efficiency across datasets.
1. Introduction
- Supervised NR-IQA depends on labeled data that is difficult and resource-intensive to acquire; KADID required approximately 300K annotations for 10,125 images.The authors identify this dependence as a constraint on supervised approaches’ scalability and broad applicability.Source: §1 · p. 1
- Prior self-supervised approaches match crops from the same distorted image, which can yield content-dependent embeddings and scatter representations of some distortions.ARNIQA’s KADID t-SNE visualization shows more discernible distortion clusters than Re-IQA [36].Source: Figure 1 · p. 1, §1 · p. 2
- ARNIQA maximizes similarity between crops from different images degraded equally, producing similar representations across content and modeling the distortion manifold.A simple linear regressor maps image representations to quality scores without fine-tuning the encoder.Source: §1 · p. 2
- ARNIQA achieves state-of-the-art performance on datasets with synthetic and authentic distortions, while improving data efficiency, generalization, and robustness compared with baselines.It requires only up to 0.5% of competitors’ training images; cross-dataset evaluation and gMAD demonstrate its generalization and robustness.Source: §1 · p. 2
- The paper introduces an image degradation model that randomly assembles ordered sequences of distortions; this yields 1.9·10^9 distinct possible compositions for synthetically degrading images.Source: §1 · p. 2
2. Related Work
Prior NR-IQA work ranges from handcrafted and supervised approaches to self-supervised representation learning, while degradation models differ in distortion coverage and composition. ARNIQA combines self-supervision on unlabeled data with randomly composed ordered degradation sequences.
2. Related Work
- Traditional methods use handcrafted features and regression, codebook methods derive quality-aware patch features, and supervised learning has recently boosted NR-IQA performance.The related-work discussion names BRISQUE and NIQE as traditional examples and CORNIA and HOSA as codebook-based examples.Source: §2 · p. 2
- Su et al. learns the distortion manifold supervised and requires distortion-specific training information, limiting its use on authentic degradations; ARNIQA uses unlabeled data.The passage also notes supervised methods’ dependence on ground-truth quality scores, which are expensive and time-consuming to collect.Source: §2 · p. 3
- Existing self-supervised methods match crops from the same distorted image, whereas ARNIQA matches patches from distinct images degraded equally and uses a frozen encoder with linear regression.QPT uses a quality-aware contrastive loss, CONTRIQUE frames representation learning as classification, and Re-IQA uses quality- and content-aware encoders.Source: §2 · p. 3
- Existing degradation models differ in distortion types and composition: RealESRGAN applies distortions from four groups in a fixed order, Re-IQA applies one of 25 types, and QPT uses nine types in three groups.ARNIQA randomly composes ordered sequences across 24 distortion types in seven groups, giving 100 times more possible compositions than existing methods.Source: §2 · p. 3
3. Proposed Approach
ARNIQA trains image representations to reflect degradation patterns rather than image content, using synthetic ordered distortion compositions and contrastive learning. Its strategy pairs equally degraded crops from different images and uses half-scale crops as hard negatives.
3. Proposed Approach
- After training, the projector is discarded, and frozen encoder features are mapped to quality scores with a linear regressor.The approach uses a pre-trained ResNet-50 encoder and a two-layer MLP projector during training.Source: §3 · p. 3
3.1. Image Degradation Model
- The degradation model composes ordered sequences from 24 distortion types in seven groups, sampling at most one degradation per group and shuffling their order.Each distortion has five intensity levels; the model can also leave images pristine with probability pprist.Source: §3.1 · pp. 3–4
- With Ndist = 4, the model yields 1.9 · 10^9 possible compositions, about 100 times the 2 · 10^7 available with QPT [58].Lower intensity levels are sampled more often, focusing finer-grained coverage on less severe patterns considered more likely in real-world scenarios.Source: §3.1 · p. 4
3.2. Training Strategy
- ARNIQA maximizes embedding similarity for crops from different images degraded in the same manner, encouraging representations of matching degradation patterns despite varying content.The learned embeddings are intended to place images degraded in the same way near one another in the distortion manifold.Source: §3.2 · pp. 4–6
- Half-scale crops serve as hard negatives: they share image content with full-scale crops but differ by the added downsampling degradation.The approach treats these crops as challenging negatives, unlike CONTRIQUE [25], which treats half-scale crops as positive examples in the same distortion class.Source: Figure 3 · p. 5, §3.2 · pp. 5–6
- The training strategy uses SimCLR-style contrastive learning, with two views across two scales producing 4B elements for a batch of size B.The loss pulls each view toward its corresponding view and away from the other full- and half-scale views.Source: §3.2 · pp. 5–6
4. Experimental Results
ARNIQA performs competitively or better across synthetic and authentic IQA datasets, with stronger cross-dataset generalization, robustness, and data efficiency than competing methods. Ablations indicate that degradation variety and training on equally degraded crops from different images matter to its performance.
4.1. Implementation Details
- The encoder starts from pretrained ResNet-50 weights and is fine-tuned for 10 epochs; evaluation uses a Ridge regressor without encoder fine-tuning.Training uses 140K pristine KADIS images, discarded degraded examples, and the paper's degradation model to synthesize training distortions.Source: §4.1 · p. 6, §4.2 · p. 6, §4.3 · p. 6
4.2. Datasets
- Evaluation covers four synthetic-distortion datasets and two authentic-distortion datasets, including FLIVE with nearly 40K real-world images and SPAQ with 11K high-resolution images.The synthetic datasets are LIVE, CSIQ, TID2013, and KADID; the authentic datasets are FLIVE and SPAQ.Source: §4.2 · p. 6
4.3. Evaluation Protocol
- The protocol measures prediction monotonicity with SRCC and accuracy with PLCC, using reference-image-based 70/10/20 splits and median results over 10 runs.FLIVE uses its official train-test split; test features combine full- and half-scale representations and average predictions across five crops per scale.Source: §4.3 · pp. 6–7
4.4. Results
- ARNIQA achieves the best average results across synthetic and authentic distortions, with performance comparable to or better than CONTRIQUE and Re-IQA.It trains on 2.8M images versus CONTRIQUE's 65M and Re-IQA's 512M and 38M encoder examples, using up to 0.5% as many examples as competitors.Source: Table 1 · p. 7, §4.4 · p. 7
- Cross-dataset SRCC results show ARNIQA significantly outperforming competing methods, especially from datasets with few distortion types to those with many.The authors hypothesize that consistent mapping from distinct distortion-manifold regions to quality scores explains this generalization.Source: §4.4 · p. 7, Table 2 · p. 7
- In gMAD tests, Re-IQA found no obvious quality differences when ARNIQA defended, while ARNIQA exposed Re-IQA failures when attacking.Figure 4 compares both models fixed at low- and high-quality levels.Source: Figure 4 · p. 8, §4.4 · p. 8
4.5. Ablation Studies
- The RealESRGAN degradation model performed poorly; uniform intensity sampling, one distortion, and seven distortions also hampered training.The authors attribute these outcomes to limited pattern variety, coarse modeling of likely real-world degradations, or distortions that are absent or too strong.Source: §4.5 · p. 8, Table 3 · p. 8
- Training on two crops from the same degraded image led to poor performance, while ensuring hard negatives in each batch improved results over random negatives.The same-image approach can work but uses more convoluted strategies, such as multiple loss terms or two encoders.Source: §4.5 · p. 8, Table 3 · p. 8
5. Conclusion
ARNIQA learns the distortion manifold using ordered synthetic degradation compositions and content-invariant training on equally degraded crops. It achieves state-of-the-art performance across synthetic and authentic distortions, alongside improved data efficiency, generalization, and robustness.
5. Conclusion
- ARNIQA randomly assembles ordered distortion sequences, enabling about 100 times more possible compositions than competing methods.It maximizes similarity between embeddings of crops from distinct images degraded equally, regardless of content, so a linear regressor can map representations to quality scores.Source: §5 · pp. 8–9
- Experiments show state-of-the-art performance on synthetic and authentic distortions, with improved generalization, data efficiency, and robustness over baselines.The authors identify blind image restoration as a subject for future study using the learned distortion manifold.Source: §5 · p. 9
6. Analysis on Data Efficiency
ARNIQA achieves state-of-the-art results while using substantially fewer training examples than competing methods. The paper reports 2.8M training images for ARNIQA, compared with 65M for CONTRIQUE and 550M for Re-IQA across its two encoders.
6. Analysis on Data Efficiency
- 4.3% and 0.5% of the training examples used by CONTRIQUE and Re-IQA, respectively, still yielded state-of-the-art results on several IQA datasets.The respective training totals are 65M for CONTRIQUE, 512M for Re-IQA's content-aware encoder, and 38M for its quality-aware encoder, compared with ARNIQA's 2.8M.Source: §6 · p. 12
7. Additional Experimental Results
Additional experiments test ARNIQA with reference images, probe regressor robustness, compare robustness through gMAD, and visualize the learned distortion manifold.
7.1. Full-Reference Image Quality Assessment
- On synthetic-distortion FR-IQA datasets, ARNIQA is competitive despite its NR-IQA design, and reference information improves performance over NR-IQA without encoder fine-tuning.The evaluation is limited to synthetic-distortion datasets because realistic-distortion datasets lack reference images.Source: §7.1 · p. 12, Table 4 · p. 13
7.2. Regressor Regularization Coefficient
- On KADID validation SRCC, ARNIQA’s best-to-worst variation across ridge coefficients is considerably lower than Re-IQA’s, indicating greater coefficient robustness.The reported values are medians over 10 random training/validation/test splits.Source: §7.2 · p. 12, Table 5 · p. 13
7.3. gMAD Competition
- In gMAD, ARNIQA exposed CONTRIQUE’s quality failures more clearly as attacker, while CONTRIQUE found no clear disparity against ARNIQA at low quality; ARNIQA was more robust.The comparison fixes each model at low- and high-quality levels in turn.Source: §7.3 · p. 13, Figure 5 · p. 13
7.4. Manifold Visualization
- ARNIQA’s UMAP shows smoother transitions than Re-IQA: combined points move toward the white-noise cluster as its intensity rises, consistent with noise being applied last.The final degradation is more visible because subsequent degradations do not modify it.Source: §7.4 · p. 14, Figure 6 · p. 14
8. Image Degradation Model
The degradation model generates varied distortion compositions and covers 24 degradation types across seven groups, with five intensity levels per type.
8.1. Distortion Compositions
- KADIS examples show that the proposed degradation model produces a wide variety of distortion patterns, which the authors say enables learning the image distortion manifold.Figure 7 shows pristine images alongside their distorted versions.Source: §8.1 · p. 14, Figure 7 · p. 15
8.2. Distortion Types
- The model covers 24 degradation types across seven KADID-defined groups, with five increasing intensity levels per type, spanning brightness, blur, spatial, noise, color, compression, and sharpness/contrast.The described operations include blur kernels, noise additions, color changes, and compression.Source: §8.2 · pp. 14–15