PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Generalized Image Quality Assessment: Relaxing the Perfect Reference Quality Assumption

Du Chen, Tianhe Wu, Kede Ma, Lei Zhang

arXiv:2503.11221v2cs.CV

TL;DR

Standard FR-IQA assumes perfect references, although imaging limitations and generative enhancement challenge that premise. The paper builds DiffIQA and SRIQA-Bench and proposes A-FINE, which adaptively combines fidelity and naturalness. Experiments report that A-FINE outperforms standard FR-IQA models across standard datasets and the new benchmarks.

  • Problem

    FR-IQA assumes perfect-quality references, but imaging limitations make such references difficult to capture and enhancement can produce better-quality images.

    Source: §1 · p. 2

  • Method

    The paper constructs DiffIQA and SRIQA-Bench and develops A-FINE to adaptively combine image fidelity and naturalness.

    Source: §1 · p. 2, §4.1 · p. 5

  • Results

    A-FINE outperforms existing methods under perfect and imperfect reference conditions, with the strongest generalization to SRIQA-Bench.

    Source: §1 · p. 2, §5.3 · p. 8

  • Takeaways & Limitations

    The introduced data and model provide resources for studying generalized FR-IQA in the era of deep generative models.

    Source: §6 · p. 8

    When reference quality is poor, A-FINE’s fidelity term could introduce significant bias in quality prediction.

    Source: §6 · p. 8

Abstract

from arXiv · show

Full-reference image quality assessment (FR-IQA) generally assumes that reference images are of perfect quality. However, this assumption is flawed due to the sensor and optical limitations of modern imaging systems. Moreover, recent generative enhancement methods are capable of producing images of higher quality than their original. All of these challenge the effectiveness and applicability of current FR-IQA models. To relax the assumption of perfect reference image quality, we build a large-scale IQA database, namely DiffIQA, containing approximately 180,000 images generated by a diffusion-based image enhancer with adjustable hyper-parameters. Each image is annotated by human subjects as either worse, similar, or better quality compared to its reference. Building on this, we present a generalized FR-IQA model, namely Adaptive Fidelity-Naturalness Evaluator (A-FINE), to accurately assess and adaptively combine the fidelity and naturalness of a test image. A-FINE aligns well with standard FR-IQA when the reference image is much more natural than the test image. We demonstrate by extensive experiments that A-FINE surpasses standard FR-IQA models on well-established IQA datasets and our newly created DiffIQA. To further validate A-FINE, we additionally construct a super-resolution IQA benchmark (SRIQA-Bench), encompassing test images derived from ten state-of-the-art SR methods with reliable human quality annotations. Tests on SRIQA-Bench re-affirm the advantages of A-FINE. The code and dataset are available at https://tianhewu.github.io/A-FINE-page.github.io/.

1. Introduction

Standard FR-IQA assumes perfect references, an assumption challenged by real imaging limitations and generative enhancement. The paper introduces DiffIQA and A-FINE to evaluate images when tests may be better than their references.

1. Introduction

  • Digital imaging limitations make perfect-quality natural-scene references difficult to capture, while generative enhancement can rescue or improve image quality.The paper notes natural scenes’ spatiotemporal complexity, high dynamic range, and wide color spectrum.Source: §1 · p. 2
  • Imperfect references undermine standard FR-IQA reliability: they cannot assess better-quality tests and may favor a worse test closer to the reference.The paper illustrates this with reference D, higher-quality tests B and C, and equally poor tests E and F.Source: §1 · p. 2
  • Prior imperfect-reference studies were limited and focused on simple synthetic scenarios whose enhancers often did not improve perceptual quality.The paper’s contributions include a generalized model and evaluations on standard datasets, DiffIQA, and SRIQA-Bench.Source: §1 · p. 2
  • DiffIQA contains approximately 180,000 enhanced images labeled worse, similar, or better than their references, and A-FINE adaptively combines fidelity and naturalness.The paper also introduces SRIQA-Bench, comprising 1,000 images from ten SR methods with complete paired-comparison annotations.Source: §1 · p. 2

2. Related Work

Conventional FR-IQA datasets typically pair references treated as perfect with synthetic distortions and collect subjective quality ratings. Work on enhanced images has explored several alternatives, but the paper characterizes these methods as constrained and liable to struggle on its benchmarks.

2. Related Work

  • Conventional FR-IQA datasets select “perfect” references, apply synthetic distortions at varied intensities, and use subjective testing to obtain MOS ground truth.Examples include LIVE, CSIQ, TID2013, and KADID-10K, which differ in distortion coverage and rating procedures.Source: §2 · p. 3
  • Standard FR-IQA methods evolved from error visibility toward structural and feature-based similarity, but remain oriented toward comparing tests with perfect-quality references.The passage discusses MSE/PSNR, VDP, MAD, LPIPS, SSIM, DISTS, and related methods.Source: §2 · p. 3
  • Methods for enhanced images include VIF, PCQI, specialized interpolation and denoising models, and CKDN, but the paper says they struggle on DiffIQA and SRIQA-Bench.These approaches were developed and tested in relatively simple, constrained scenarios; A-FINE is designed for both perfect and imperfect-reference conditions.Source: §2 · p. 3

3. Proposed Dataset: DiffIQA

DiffIQA is constructed by generating diverse enhanced images and collecting relative-quality judgments. Its diffusion-based enhancer produces outputs of varying perceived quality, while the resulting dataset includes large-scale annotations and better-than-reference examples.

3. Proposed Dataset: DiffIQA

  • DiffIQA construction combines test-image generation by a diffusion-based enhancer with subjective testing for quality annotations.The paper presents these as the dataset’s two construction stages.Source: §3 · p. 3

3.1. Generative Image Enhancer

  • The PASD-based enhancer uses Stable Diffusion, ControlNet, and pixel-aware cross-attention, with slight blind degradations applied to half its training inputs.The trainable components predict noise added to the input latent, and the degradations help produce worse-, similar-, and better-quality outputs.Source: §3.1 · pp. 3–4

3.2. Construction of DiffIQA

  • The dataset uses 29,868 inputs to generate 179,208 test images, with subjects judging each reference–test pair as worse, similar, or better in incomplete paired comparisons.Each pair received at least three annotations; the study involved 240 subjects.Source: §3.2 · p. 4

3.3. DiffIQA Statistics

  • After filtering, DiffIQA retained 76,515 worse, 24,654 similar, and 76,150 better labels; 42.49% of retained judgments rated tests better than references.The dataset’s stated distinguishing features include its scale, diffusion-based distortions, and inclusion of better-than-reference test images.Source: §3.3 · p. 4

4. Proposed FR-IQA Model: A-FINE

A-FINE is a generalized FR-IQA model that combines fidelity and test-image naturalness without assuming the reference is higher quality. Its adaptive weighting is trained through pairwise quality ranking.

4. Proposed FR-IQA Model: A-FINE

  • A-FINE generalizes FR-IQA to references that may be lower quality than the test image.Its system uses a shared feature transformation for fidelity and naturalness measurements that are adaptively combined into a quality score.Source: §4 · p. 4, §4.1 · p. 5, Figure 5 · p. 5

4.1. Computation of A-FINE

  • A-FINE combines fidelity F(x, y) with test-image naturalness N(y), increasing naturalness's influence when the test appears more natural than the reference.It uses a shared CLIP ViT backbone to compute fidelity and naturalness features.Source: §4.1 · pp. 5–6

4.2. Training Procedure of A-FINE

  • A-FINE is optimized by pairwise ranking of same-content test-image pairs, with labels distinguishing better, tied, and worse quality relative to the reference.The training procedure models perceived quality as Gaussian with fixed variance and uses a fidelity loss for end-to-end optimization.Source: §4.2 · p. 6

5. Experiments

A-FINE is evaluated against standard and generalized FR-IQA models on established datasets, DiffIQA, and SRIQA-Bench. Results favor A-FINE overall and on enhanced-image assessment, while ablations examine its backbone, training data, and training strategy.

5.1. Experimental Setups

  • A-FINE is compared with nine standard and four generalized FR-IQA models, including original and, where applicable, fine-tuned versions.Its training combines DiffIQA with TID2013, KADID-10K, and PIPAL and uses three phases to train naturalness, fidelity, then adaptive weighting and mappings.Source: §5.1 · pp. 6–7

5.2. SRIQA-Bench

  • SRIQA-Bench uses 100 original images degraded into low-resolution inputs and ten SR methods to generate test images.Paired comparisons were assessed by at least ten subjects per pair; 40 subjects participated.Source: §5.2 · p. 7

5.3. Main Results

  • A-FINE achieves the highest average results across the TID2013, KADID-10K, PIPAL, and DiffIQA test sets, attributed to adaptive weighting of fidelity and naturalness.Fine-tuned models generally improve on DiffIQA but show a performance drop on standard IQA datasets.Source: §5.3 · p. 7, Table 2 · p. 7
  • A-FINE demonstrates the strongest generalization to SRIQA-Bench, while fine-tuned models improve performance on generation-based super-resolution outputs.Most models perform adequately on regression-based SR methods, whose outputs are described as clearly inferior to their references.Source: §5.3 · pp. 7–8, Table 3 · p. 8

5.4. Ablation Studies

  • A-FINE benefits from stronger pretrained backbones, and training on combined DiffIQA and standard datasets yields the best average performance and cross-dataset generalization.The three-phase strategy stabilizes training dynamics and improves generalization compared with single-phase training.Source: §5.4 · p. 8, Table 4 · p. 8, Table 5 · p. 8

6. Conclusion and Limitations

The paper frames generalized FR-IQA as a problem for deep generative imaging and presents DiffIQA, SRIQA-Bench, and A-FINE as data and model contributions. It also identifies boundaries and open questions for A-FINE.

6. Conclusion and Limitations

  • The paper introduces DiffIQA and SRIQA-Bench for training and testing generalized FR-IQA, and A-FINE to adaptively combine fidelity and naturalness.The authors hope these contributions will encourage research on generalized FR-IQA in the era of deep generative models.Source: §6 · p. 8
  • A-FINE can perform effectively when reference quality is reasonably high, but poor-quality references may bias its fidelity term.The optimal functional form for asymmetric distance measures remains open, as does the best trade-off between assessing distorted and enhanced images.Source: §6 · p. 8

Appendix

  • The appendix covers enhancer training and inference, subjective-testing setups for DiffIQA and SRIQA-Bench, and discussion of generated fake details.Source: Appendix · p. 12

A.1. Architecture

The enhancer combines image-derived control features with a diffusion model. Its training uses noise prediction as the objective and updates only selected modules.

A.1. Architecture

  • The enhancer’s training architecture uses a lightweight convolutional network to produce a content-aware ControlNet signal for the diffusion UNet.The original image or its degraded version is encoded into a latent representation, then Gaussian noise is added for the UNet input.Source: §A.1 · p. 12, Figure 6 · p. 12
  • Pixel-aware cross-attention lets the ControlNet signal interact with the diffusion UNet during noise prediction.Source: §A.1 · p. 12
  • Training minimizes the difference between predicted and added Gaussian noise, updating only the convolutional network, ControlNet, and PACA modules.Source: §A.1 · p. 12

A.2. Training Specifications

The enhancer’s training diversifies input quality with blind degradations, while inference varies degradation, latent noise, and sampling steps. The appendix also reports its training scale and inference throughput.

A.2. Training Specifications

  • Half of the original inputs receive slight blind degradations to diversify the enhancer’s output quality.Degradation blurs and resizes the original image, adds Gaussian or Poisson noise, and applies JPEG compression; the result is resized to the original resolution before enhancement.Source: §A.2 · p. 12
  • Enhancer training used eight NVIDIA V100 GPUs, 100,000 iterations, a fixed learning rate of 5 × 10^-5, and 512 × 512 images.Source: §A.2 · p. 12

B. Inference Details of the Generative Image Enhancer

C. Subjective Experimental Setups of DiffIQA

DiffIQA subjective testing used a GUI for paired reference–test judgments, with training on image naturalness and visible distortion. The protocol limited viewing time and allowed subjects to pause and resume.

C. Subjective Experimental Setups of DiffIQA

D. Subjective Experimental Setups of SRIQA-Bench

SRIQA-Bench subjects first judged fidelity to the reference, then used perceived quality when fidelity was comparable. Testing instructions also prioritized fake or deformed details that affected fidelity.

D. Subjective Experimental Setups of SRIQA-Bench

E. Discussions on the Generated “Fake” Details

  • Participants were instructed to treat plausible but hallucinated details as a reason for a worse-quality judgment when they harmed fidelity.In the example, sharper generated content coexists with visibly distorted text; A-FINE evaluates the reference as better, matching human judgments.Source: E. Discussions on the Generated “Fake”… · p. 14, Figure 11 · p. 15
  • DiffIQA examples group generated images as worse, similar, or better quality than their references.Source: Figure 10 · p. 15
Loading 2503.11221v2…