VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rank

Tianhe Wu; Jian Zou; Jie Liang; Lei Zhang; Kede Ma

Source-linked AI summary

VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rank

Tianhe Wu, Jian Zou, Jie Liang, Lei Zhang, Kede Ma

arXiv:2505.14460v2May 20, 2025cs.CV

TL;DR

NR-IQA needs reasoning-based methods that can assess relative visual quality without the annotation and rigidity limitations of supervised fine-tuning. VisualQuality-R1 uses reinforcement learning to rank with GRPO-generated quality estimates, Thurstone comparisons, and continuous fidelity rewards; it consistently outperforms strong NR-IQA baselines while producing rich descriptions and supporting multi-dataset training without scale realignment.

Problem
Reasoning-induced computation remains underexplored for IQA, while SFT-based VLM methods require costly annotations and may overfit biases, forget pretrained knowledge, or produce rigid outputs.
Method
VisualQuality-R1 applies RL2R with GRPO to generate multiple scores per image, uses Thurstone comparative probabilities, and defines rewards through continuous fidelity measures.
Results
VisualQuality-R1 consistently outperforms discriminative NR-IQA models and a reasoning-induced baseline, while improving multi-dataset image-generation results from 0.777/0.814 to 0.791/0.831 SRCC/PLCC.
Takeaways & Limitations
The model provides human-aligned quality descriptions for downstream feedback and quality control, and supports multi-dataset training without perceptual scale realignment.
Takeaways & Limitations
As a test-time-scaling method, VisualQuality-R1 is slow, expensive, and memory-hungry and may compound early errors into confidently wrong predictions.

Abstract

from arXiv · show

DeepSeek-R1 has demonstrated remarkable effectiveness in incentivizing reasoning and generalization capabilities of large language models (LLMs) through reinforcement learning. Nevertheless, the potential of reasoning-induced computation has not been thoroughly explored in the context of image quality assessment (IQA), a task depending critically on visual reasoning. In this paper, we introduce VisualQuality-R1, a reasoning-induced no-reference IQA (NR-IQA) model, and we train it with reinforcement learning to rank, a learning algorithm tailored to the intrinsically relative nature of visual quality. Specifically, for a pair of images, we employ group relative policy optimization to generate multiple quality scores for each image. These estimates are used to compute comparative probabilities of one image having higher quality than the other under the Thurstone model. Rewards for each quality estimate are defined using continuous fidelity measures rather than discretized binary labels. Extensive experiments show that the proposed VisualQuality-R1 consistently outperforms discriminative deep learning-based NR-IQA models as well as a recent reasoning-induced quality regression method. Moreover, VisualQuality-R1 is capable of generating contextually rich, human-aligned quality descriptions, and supports multi-dataset training without requiring perceptual scale realignment. These features make VisualQuality-R1 especially well-suited for reliably measuring progress in a wide range of image processing tasks like super-resolution and image generation.

1 Introduction

NR-IQA seeks human-aligned image-quality judgments without pristine references, but VLM-based approaches face annotation, overfitting, forgetting, and output-rigidity challenges. VisualQuality-R1 addresses these issues with reinforcement learning to rank and supports rich descriptions and multi-dataset training without perceptual scale realignment.

NR-IQA measures image quality without requiring pristine reference images, supporting real-world scenarios where references are unavailable.
VLMs can combine low-level distortions with high-level perceptual attributes to produce nuanced quality descriptions with stronger generalization.
SFT-based NR-IQA requires labor-intensive quality-description annotation and can overfit data biases, forget pretrained knowledge, and produce rigid outputs.
VisualQuality-R1 trains a reasoning-induced NR-IQA model with RL2R, using GRPO, Thurstone comparative probabilities, and continuous fidelity rewards for relative quality ranking.
VisualQuality-R1 outperforms discriminative NR-IQA and reasoning-induced regression methods while generating human-aligned descriptions and supporting multi-dataset training without scale realignment.

2 Related Work

NR-IQA research has progressed from handcrafted and deep regression models toward ranking-based and VLM-based approaches. Recent VLM methods capture contextual semantics but commonly rely on supervised fine-tuning, while reasoning-induced models represent a newer direction.

Regression-based Models: Early regression-based NR-IQA used handcrafted natural-scene, degradation-specific, or human-visual-system-inspired features.
Regression-based Models: Deep regression models later used end-to-end neural networks to predict quality scores or distributions, becoming the dominant paradigm.
Ranking-based Models: Ranking-based NR-IQA models treated visual quality as intrinsically relative, addressing shortcomings of absolute-quality regression approaches.
VLM-based Models: VLM-based NR-IQA methods leverage multimodal representation learning to capture contextual semantics and generate scores, distributions, or descriptions.
VLM-based Models: Recent VLM-based methods include multitask CLIP adaptation and supervised fine-tuning approaches such as Q-Align, Compare2Score, DepictQA, and DeQA-Score.

3 Reasoning-Induced NR-IQA

VisualQuality-R1 is an NR-IQA model trained with RL2R, integrating GRPO-generated quality estimates with Thurstone-model ranking and continuous fidelity rewards. It produces reasoning-based scores and descriptions while supporting multi-dataset training without perceptual scale realignment.

VisualQuality-R1 via RL2R: VisualQuality-R1 integrates the Thurstone model within GRPO to train an NR-IQA model through reinforcement learning to rank.The method generates multiple quality estimates and compares images using their predicted quality distributions.
VisualQuality-R1 via RL2R: For each image, GRPO generates K quality predictions and uses their mean and variance to represent predictive uncertainty.The model computes asymmetric probabilities for ordered image pairs using standardized score differences.
VisualQuality-R1 via RL2R: Comparative probabilities subtract one image’s mean predicted score from another estimate and normalize the difference by their sample variances.A small positive constant prevents division by zero, while sample means help penalize outlier predictions and stabilize reward calculation.
VisualQuality-R1 via RL2R: RL2R defines each estimate’s reward using a continuous fidelity measure derived from human preferences rather than a discretized binary reward.The resulting continuous feedback captures subtle ranking variations and is standardized within each generated-response group for policy optimization.
VisualQuality-R1 via RL2R: The structured prompt requests a 1-to-5 quality score, step-by-step reasoning in <think> tags, and a final score in <answer> tags.The score is specified as a float rounded to two decimal places.
VisualQuality-R1 via RL2R: VisualQuality-R1 is presented as supporting rich textual justifications, multi-dataset training without perceptual scale realignment, and reduced SFT-related scalability and overfitting issues.Its regularized objective constrains policy updates through KL regularization, clipping, and a balancing coefficient β.

4 Experiments

Experiments show that VisualQuality-R1 performs strongly across distortion scenarios, benefits from continuous-fidelity RL2R and uncertainty-aware Thurstone comparisons, and supports stable multi-dataset training. Training also progressively improves prediction stability and visual reasoning, while reducing GRPO responses offers a cost–accuracy trade-off.

Main Results: VisualQuality-R1 achieves the best average results among evaluated NR-IQA models across distortion scenarios.VLM-based models outperform traditional and discriminative deep-learning models, while reasoning-induced models surpass SFT-based counterparts.
Main Results: 0.791/0.831 average SRCC/PLCC rises from 0.777/0.814 with KADID-10K and SPAQ training.This multi-dataset training does not require perceptual scale realignment, although performance dips slightly for image generation.
Ablation Studies: Reducing GRPO responses from six to four or five has only a marginal effect on accuracy.The result indicates a favorable trade-off between computational cost and accuracy.
Ablation Studies: Continuous fidelity rewards generalize better than binary rewards within the same RL2R framework.Both reward variants outperform the regression-based Q-Insight model.
Ablation Studies: Averaging quality scores rather than probabilities yields higher performance across distortion scenarios.Fixing variances to one degrades performance on nearly all datasets, indicating that sample variances improve comparison reliability and reward stability.
Further Analysis: Prediction variability falls steadily across four datasets as training progresses.VisualQuality-R1's visual reasoning also matures from superficial judgments toward more balanced, cautious assessments of processing artifacts.

5 Conclusion and Discussion

VisualQuality-R1 uses RL2R to improve reasoning-based NR-IQA, producing more precise quality estimates and human-aligned explanations. The approach performs strongly but remains computationally expensive and can compound early reasoning errors.

Conclusion and Discussion: VisualQuality-R1 integrates the Thurstone model with GRPO and continuous fidelity rewards to improve relative ranking and policy-gradient precision.Multiple quality predictions encode uncertainty, while continuous rewards avoid discretized binary supervision.
Conclusion and Discussion: Later training stages detect stylization, blur, and filtering effects in a super-resolved image, yielding more accurate scores and human-aligned justifications.Early iterations overlook artifacts and overestimate quality.
Conclusion and Discussion: The model generates contextually rich, human-aligned descriptions that support transparency, interpretation, and downstream quality diagnosis.Reported applications include content filtering, local enhancement prioritization, and quality-aware image retrieval.
Conclusion and Discussion: VisualQuality-R1 consistently surpasses discriminative NR-IQA methods and a reasoning-induced baseline across extensive experiments.The experiments also report robustness across synthetic and realistic distortions.
Conclusion and Discussion: As a test-time scaling approach, VisualQuality-R1 is slow, expensive, memory-hungry, and vulnerable to compounding early errors.The paper suggests adaptive reasoning, rationale compression or distillation, and self-consistency sampling as future directions.

A Generalization Probing via gMAD Competition

The gMAD competition probes cross-dataset generalization by having models attack and defend against perceptual inconsistencies. VisualQuality-R1 exposes flaws in Q-Insight and remains closely aligned with human judgments under attack.

Generalization Probing via gMAD Competition: gMAD evaluates model generalization by using adversarial image pairs selected where models disagree about relative quality.Human observers assess the selected pairs, enabling targeted evaluation without exhaustive ground-truth labeling.
Generalization Probing via gMAD Competition: When attacked, VisualQuality-R1 demonstrates strong robustness and aligns closely with human perception of image quality.Its dual role combines aggressive flaw detection with resistance to adversarial challenges.

B Model Complexity Comparison

VisualQuality-R1 has higher inference costs than discriminative NR-IQA models and small-scale VLMs. The paper frames this overhead as a deliberate choice supporting fine-grained reasoning and robust quality rating across diverse distortions.

B Model Complexity Comparison: VisualQuality-R1 incurs higher inference costs than MUSIQ, MANIQA, and LIQE.The comparison includes discriminative models and the small-scale VLM LIQE.
B Model Complexity Comparison: The increased computational demand reflects a deliberate design choice for fine-grained quality reasoning.
B Model Complexity Comparison: The design aims to support robust quality rating across diverse distortion scenarios.

C Additional Visual Reasoning Examples

Additional examples illustrate VisualQuality-R1’s reasoning under synthetic and realistic distortions, alongside a gMAD comparison with Q-Insight. The examples also expose occasional scores outside the prescribed range.

Additional Visual Reasoning Examples: The additional examples cover visual reasoning under both synthetic and realistic distortion scenarios.Figures 6 and 7 provide samples for the two settings.
Additional Visual Reasoning Examples: Figure 5 presents gMAD competitions between VisualQuality-R1 and Q-Insight with each model fixed at low- and high-quality levels.The four panels vary which model serves as the fixed defender and its quality level.
Additional Visual Reasoning Examples: In the synthetic example, the model links out-of-focus regions and reduced detail to a quality score of 1.00.The reasoning identifies blurred rocks, trees, and water as degrading scene perception.
Additional Visual Reasoning Examples: In the realistic example, the model balances nighttime lighting against visible architecture and assigns a quality score of 3.00.It notes adequate exposure and the absence of significant noise or blur.