PaperlayerView exact revision on arXiv

Source-linked AI summary

From Glance to Scrutiny: Progressive Distortion Reasoning for Fine-Grained Image Quality Assessment

Aoting Zhang, Mingze Gao, Dongbao Yang, Longyi Chen, Daoxin Zhang, Yi Wu, Yao Hu, Yu Zhou

arXiv:2609.08316v1cs.CV

TL;DR

Holistic IQA approaches offer limited insight into where distortions occur and how they affect perceived quality, despite the need to analyze localized, heterogeneous degradations. GS-IQA treats IQA as progressive region-level diagnosis and trains it with dependency-aware reinforcement learning and targeted degradation synthesis. Experiments report stronger fine-grained diagnosis than existing MLLM-based IQA methods and transfer to conventional global quality prediction.

  • Problem

    Holistic quality prediction provides limited evidence about distortion locations and their effects, hindering fine-grained analysis of localized and heterogeneous degradations.

    Source: §1 · pp. 1–2

  • Method

    GS-IQA diagnoses each degraded region through Where–What–How reasoning, using gated severity rewards after localization and recognition and online reward-conditioned generation for difficult severity cases.

    Source: §1 · pp. 2–3

  • Results

    GS-IQA consistently outperforms existing MLLM-based IQA methods in fine-grained diagnosis and transfers to conventional global quality prediction.

    Source: §1 · p. 3, §5 · p. 11

  • Takeaways & Limitations

    Diag-Bench provides region-level evaluation with 25K samples spanning 12 distortion types and five ordinal severity levels.

    Source: §1 · p. 3

Abstract

from arXiv · show

Multi-modal large language models (MLLMs) have demonstrated significant potential in image quality assessment (IQA) by bridging visual perception with descriptive evaluations. However, existing approaches mainly focus on holistic quality prediction, often functioning as black boxes that provide limited insight into where distortions occur and how they affect perceived quality, hindering fine-grained analysis of localized and heterogeneous degradations. We propose GS-IQA, a framework that reformulates IQA as a progressive Where--What--How diagnosis, emulating the human perceptual process from an initial glance to closer scrutiny. Since a severity judgment is meaningful only for a correctly localized and recognized region, we realize this progression through a two-stage reinforcement learning paradigm that respects such dependencies: the glance stage uses a perception-gated reward to establish where degradations lie and what they are, activating severity feedback only once both are correct, while the scrutiny stage introduces online reward-conditioned degradation generation to synthesize hard examples targeted at the model's perceptual bottlenecks, sharpening its discrimination of subtle severity variations. To enable systematic evaluation, we construct Diag-Bench, a region-level IQA benchmark of about 25K curated samples spanning 12 distortion types and five ordinal severity levels. Extensive experiments show that GS-IQA consistently surpasses state-of-the-art methods in distortion localization, recognition, and severity estimation, and that its diagnostic representations transfer effectively to conventional global quality prediction across diverse external benchmarks. Code and data will be released.

1 INTRODUCTION

GS-IQA reframes image quality assessment as progressive region-level diagnosis, linking each degraded area to its distortion type and severity. It pairs this approach with dependency-aware reinforcement learning and Diag-Bench, a benchmark for systematic evaluation.

1 INTRODUCTION

  • Holistic score regression and image-level summaries provide limited evidence about distortion location, type, and severity, especially for spatially heterogeneous degradations.The introduction notes that such summaries can obscure artifacts relevant to distortion-aware restoration and region-adaptive enhancement.Source: §1 · pp. 1–2
  • Human quality perception progresses from noticing where and what a degradation is to scrutinizing how severely it affects quality, making severity judgments dependent on correct region and type recognition.Otherwise, severity signals may be dominated by irrelevant or pristine content.Source: §1 · p. 2
  • GS-IQA associates each degraded region with its distortion category and ordinal severity through progressive Where–What–How reasoning.The framework distinguishes localization, distortion recognition, and severity estimation rather than condensing heterogeneous degradations into one global judgment.Source: §1 · pp. 2–3
  • Perception-gated rewards establish reliable localization and recognition before severity feedback, while online reward-conditioned generation targets underperforming distortion categories.The adaptive synthesis is intended to mitigate severity-learning saturation on static data.Source: §1 · pp. 2–3
  • Diag-Bench is a region-level quality diagnosis benchmark with 25K samples spanning 12 distortion categories and five ordinal severity levels; experiments show GS-IQA’s effectiveness in fine-grained distortion assessment and transferability to conventional global quality prediction.Source: §1 · p. 3

2 RELATED WORKS

Prior IQA methods increasingly incorporate language, grounding, and reinforcement learning, but gaps remain in spatially explicit diagnosis and in training for interdependent localization, recognition, and severity. GS-IQA addresses these gaps with gated severity feedback and reward-conditioned online synthesis.

2.1 IMAGE QUALITY ASSESSMENT

  • Score-based FR and NR methods largely compress quality into a scalar, while MLLM and spatially grounded approaches add descriptions or localized evidence but leave the dependency among localization, recognition, and severity underexploited.GS-IQA uses reliable Where–What perception to constrain ordinal severity supervision.Source: §2.1 · p. 3

2.2 REINFORCEMENT LEARNING FOR MLLMS

  • In region-level diagnosis, independently optimized objectives can assign severity credit to incorrect regions or types, and fixed corpora lose informativeness as familiar degradations dominate.Existing curriculum or reweighting strategies reorganize available samples but do not expand the training distribution; GS-IQA gates severity feedback and synthesizes difficult categories online.Source: §2.2 · pp. 3–4, Figure 2 · p. 4

3 METHODOLOGY

GS-IQA frames fine-grained IQA as progressive Where–What–How diagnosis, with severity judgments conditioned on reliable localization and distortion recognition. Its Glance and Scrutiny stages support this process, and Diag-Bench provides region-level evaluation across distortion types and severity levels.

3.1 OVERVIEW OF GS-IQA

  • GS-IQA diagnoses each degraded region by localizing it, recognizing its distortion type, and estimating ordinal severity within a two-stage GRPO framework.Severity prediction is meaningful only for a spatially matched region with a correctly recognized category.Source: §3.1 · p. 4

3.2 GLANCE STAGE: PERCEPTION-GATED MULTI-TASK LEARNING

  • The Glance stage jointly learns localization and category recognition, releasing severity feedback only for spatially matched regions with correctly recognized distortion types.It uses optimal bipartite matching and retains pairs with intersection-over-union above 0.5; ordinal severity reward decreases as prediction distance grows.Source: §3.2 · pp. 4–5

3.3 SCRUTINY STAGE: ONLINE REWARD-CONDITIONED GENERATION

  • The Scrutiny stage tracks category-wise severity performance and uses online reward-conditioned synthesis to direct more training toward distortions that remain difficult to grade.The evolving sampling distribution focuses on low-reward categories while preserving exploration, and synthesized samples include region-level supervision.Source: §3.3 · p. 6

3.4 DIAG-BENCH: BENCHMARK FOR FINE-GRAINED QUALITY DIAGNOSIS

  • Diag-Bench provides 25K region-level samples covering 12 distortion categories and five ordinal severity levels, combining authentic and synthetic degradations.Its diagnosis tasks are localization, distortion identification, and ordinal severity estimation.Source: §3.4 · p. 7

4 EXPERIMENTS

Experiments evaluate GS-IQA’s regional diagnosis, transfer to external benchmarks, and the effects of its training design. GS-IQA-Scrutiny improves fine-grained diagnosis and transfers to both external diagnostic tasks and holistic quality prediction.

4.1 EXPERIMENTAL SETUP

  • Evaluation covers region-level localization, recognition, and severity; cross-dataset diagnosis without fine-tuning; and transfer to holistic quality prediction.The implementation uses Qwen3-VL-4B and two consecutive reinforcement stages without preceding supervised fine-tuning.Source: §4.1 · p. 7

4.2 COMPARISON WITH STATE-OF-THE-ART METHODS

  • GS-IQA-Scrutiny reaches SevAcc 0.732, Recall 0.909, F1 0.930, and ClsAcc 0.947 on Diag-Bench, improving over the task-aligned variants in severity estimation.Perception-gated rewards raise Glance SevAcc to 0.678; category-wise severity feedback and online sampling refine the Scrutiny stage.Source: Table 1 · p. 8, §4.2 · p. 8
  • On globally degraded images, GS-IQA-Scrutiny improves the best competing ClsAcc from 0.482 to 0.916 and SevAcc from 0.365 to 0.735.This setting removes localization and measures distortion recognition and severity estimation.Source: Table 2 · p. 8, §4.2 · p. 8

4.3 GENERALIZATION AND TRANSFERABILITY

  • Without dataset-specific fine-tuning, GS-IQA exceeds the strongest baseline’s ClsAcc by 50.3, 41.6, 29.3, and 32.8 points across four external datasets, with SevAcc gains up to 14.5 points.Evaluation maps image-level annotations to the model’s distortion categories and five-level severity labels.Source: Table 3 · p. 9, §4.3 · p. 9
  • Across five external benchmarks, GS-IQA averages 0.860 SRCC and 0.869 PLCC, improving on VisualQuality-R1 by 0.031 SRCC and 0.015 PLCC.It ranks first on BID, CLIVE, and KonIQ after fine-tuning a global quality regressor using the same protocol as VisualQuality-R1.Source: Table 4 · p. 9, §4.3 · p. 9

4.4 ABLATION STUDIES

  • Perception-gated rewards yield the highest Glance SevAcc, 0.678, while removing the gate lowers F1 from 0.895 to 0.878 and SevAcc from 0.678 to 0.651.Continuous linear and ordinal ranking rewards yield SevAcc of 0.627 and 0.304, respectively.Source: §4.4 · p. 9, Table 5 · p. 10
  • Reward-conditioned online generation achieves SevAcc 0.732, exceeding random generation, while removing Glance initialization degrades localization, recognition, and severity performance.Static-only training reaches a performance bottleneck; category-wise feedback directs additional sampling toward distortions that remain difficult to grade.Source: §4.4 · p. 9, Table 5 · p. 10
  • With 50 source images (1% of the training pool), Scrutiny reaches F1 0.902, above the static-data Glance baseline; expanding the pool to 3,775 images raises F1 to 0.930 and SevAcc to 0.732.Scaling from 4B to 32B raises F1 from 0.930 to 0.952 and SevAcc from 0.732 to 0.754; Figure 3 shows steadily rising severity rewards for representative distortions during Scrutiny.Source: Table 6 · p. 10, §4.4 · p. 10, Figure 3 · p. 10

5 CONCLUSION

GS-IQA diagnoses heterogeneous degradations by locating affected regions, identifying distortion types, and grading severity. Its two-stage reinforcement learning approach supports fine-grained diagnosis and transfers to external diagnostic benchmarks and holistic quality prediction.

  • GS-IQA characterizes heterogeneous degradations along Where–What–How dimensions instead of collapsing them into a single score.It locates affected regions, recognizes distortion types, and grades severity.Source: §5 · pp. 10–11
  • The two-stage reinforcement learning scheme gates severity rewards on correct localization and recognition before using online reward-conditioned generation to refine severity estimates.The Glance stage consolidates perception; Scrutiny supplies targeted hard cases.Source: §5 · p. 11
  • GS-IQA outperforms existing MLLM-based IQA methods in fine-grained diagnosis, generalizes without dataset-specific fine-tuning, and transfers to holistic quality prediction.The paper concludes that fine-grained diagnosis supports both local interpretation and image-level prediction.Source: §5 · p. 11
Loading 2609.08316v1…