PaperlayerView exact revision on arXiv

Source-linked AI summary

Reasoning as Representation: Rethinking Visual Reinforcement Learning in Image Quality Assessment

Shijie Zhao, Xuanyu Zhang, Weiqi Li, Junlin Li, Li Zhang, Tianfan Xue, Jian Zhang

arXiv:2510.11369v2cs.CV

TL;DR

The paper investigates why RL-trained IQA models generalize and whether costly reasoning is essential. It analyzes their compact quality-text representations and introduces RALI, which aligns images to those representations without reasoning or an LLM. RALI matches reasoning-based models’ generalization with substantially lower parameter and inference costs.

  • Problem

    The source of RL-based IQA models’ generalization is underanalyzed, while stepwise reasoning adds latency and loading overhead that limits deployment.

    Source: §1 · p. 2

  • Method

    The paper analyzes how RL-trained MLLMs map images into compact quality-reasoning text and uses contrastive learning in RALI to align images with quality text for scoring without retaining reasoning or an LLM.

    Source: §1 · p. 2

  • Results

    RALI achieves generalization on par with RL-based MLLMs, uses about 4% of Q-Insight’s parameters, and reduces inference time and memory by more than 95%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper concludes that generalization can be realized through RALI without a reasoning process or dependence on LLMs.

    Source: §1 · p. 2

    RALI’s performance ceiling is constrained by the representational and reasoning capacity of its CLIP vision encoder, and experiments primarily target natural-image IQA.

    Source: B LIMITATIONS AND BROADER IMPACTS · p. 15

Abstract

from arXiv · show

Reasoning-based image quality assessment (IQA) models trained through reinforcement learning (RL) exhibit exceptional generalization, yet the underlying mechanisms and critical factors driving this capability remain underexplored in current research. Moreover, despite their superior performance, these models incur inference energy usage and latency orders of magnitude higher than their earlier counterparts, restricting their deployment in specific scenarios. Through extensive experiments, this paper verifies and elaborates that through RL training, MLLMs leverage their reasoning capability to convert redundant visual representations into compact, cross-domain aligned text representations. This conversion is precisely the source of the generalization exhibited by these reasoning-based IQA models. Building on this fundamental insight, we propose a novel algorithm, RALI, which employs contrastive learning to directly align images with these generalizable text representations learned by RL. This approach eliminates the reliance on reasoning processes and even obviates the need to load an LLM. For the quality scoring task, this framework achieves generalization performance comparable to reasoning-based models while requiring less than 5% of their model parameters and inference time.

1 INTRODUCTION

The introduction asks how reasoning-based IQA models generalize and whether costly reasoning is necessary. It attributes their generalization to compact quality-text representations and presents RALI as a reasoning-free alternative.

1 INTRODUCTION

  • IQA supports photography selection, video-quality monitoring, and reward signals for generative-model training, where its performance affects RL convergence and strategy effectiveness.The introduction notes applications in natural settings as well as generative algorithms.Source: §1 · p. 1
  • The paper identifies two unresolved issues: the source of RL-based IQA generalization is underanalyzed, and stepwise reasoning adds latency and loading overhead that limits deployment.It asks how reasoning relates to generalization and whether reasoning is essential.Source: §1 · p. 2
  • RL-trained models shift scoring from more than 1,000 visual tokens to fewer than 100 quality-text tokens, compressing representations by more than 10 times.The paper attributes generalization to this compact representation mitigating domain discrepancies and maintaining stable alignment across domains.Source: §1 · p. 2
  • RALI achieves generalization on par with RL-based MLLMs while eliminating reasoning and LLM deployment, using about 4% of Q-Insight’s parameters.The framework also reduces inference time and memory by more than 95%.Source: Figure 1 · p. 1, §1 · p. 2

2 RELATED WORKS

Related work spans classical full-reference and no-reference IQA, then MLLM approaches that produce scores or descriptive judgments. Recent RL-based MLLM methods output both reasoning text and scores and are reported to generalize better than SFT-based methods.

Image Quality Assessment

  • Full-reference IQA compares distorted images with pristine references, using metrics such as SSIM (Wang et al. (2004)) and deep-learning metrics such as LPIPS (Zhang et al. (2018)).No-reference methods estimate perceptual quality without an explicit reference, ranging from handcrafted natural-scene statistics to learned quality priors.Source: §2 · p. 3

MLLM-Based IQA Methods

  • Recent RL-based IQA methods jointly output quality reasoning text and scores, with superior generalization to SFT-based methods.The section cites Li et al. (2025), Zhang et al. (2025a), and Wu et al. (2025b) as recent examples.Source: §2 · p. 3
  • MLLM score-based methods such as Q-Align (Wu et al. (2024b)) and DeQA-Score (You et al. (2025)) produce numerical ratings but limit intrinsic descriptive ability.Description-based approaches seek richer qualitative judgments and interpretability, while relying on large volumes of textual annotations for SFT.Source: §2 · p. 3

3 REVISITING REASONING-BASED MLLMS IN IQA

RL-trained IQA models shift scoring toward concise, quality-relevant reasoning text, which compresses visual information into a cross-domain representation. The section uses this account to motivate RACT, a framework for training across multiple IQA datasets.

3.1 PRELIMINARIES

  • Q-Insight uses GRPO with score-regression and degradation-perception tasks, generating reasoning before the score during inference.Source: §3.1 · p. 3

3.2 REASONING MECHANISMS OF MLLMS IN IMAGE QUALITY SCORING.

  • During score-token generation, Q-Insight assigns 95% of attention to prior reasoning text, while its reasoning is more concise and more correlated with quality than Qwen-VL’s.The 95% attention figure excludes the fixed prompt; Q-Insight’s reasoning is also described as more quality-relevant.Source: Figure 2 · p. 3, §3.2 · p. 4

3.3 KEY TO GENERALIZATION: COMPRESSING VISUALS INTO TEXT REPRESENTATION

  • Across KonIQ and SPAQ, visual-token distributions show a notable domain gap that is alleviated in text; cross-dataset reasoning is comparable and reduces overfitting risk.Source: §3.3 · p. 4
  • RL yields a generalizable visual-to-text compression, and text-token representational capability supports strong score generalization.For a 512×384 image, visual representations use approximately 1,000 tokens, while fewer than 100 text tokens suffice.Source: §3.3 · pp. 4–5

3.4 REASONING-ALIGNED CROSS-DOMAIN TRAINING FRAMEWORK

  • RACT independently RL-trains on each IQA dataset, creates unified image-text labels from generated reasoning, then applies cross-domain SFT; it uses score information from one dataset because multi-domain scores impair convergence.Source: §3.4 · p. 5

4 REASONING-ALIGNED LIGHTWEIGHT IQA FRAMEWORK

RALI aligns RL-generated quality reasoning text with a visual encoder through a visual-token-to-text-token-to-score pipeline. Its three stages target performance close to RL-based IQA while offering speed and memory advantages.

4 REASONING-ALIGNED LIGHTWEIGHT IQA FRAMEWORK

  • RALI follows three steps: contrastive alignment, feature compression, and scoring definition, forming a visual-token-to-text-token-to-score pipeline.The framework is designed to approach RL-based IQA performance while offering speed and memory advantages.Source: §4 · p. 5, Figure 5 · p. 6
  • The model generates image–text–score triplets with Q-Insight, diversifies reasoning texts with different seeds, then trains the CLIP image encoder while freezing its text encoder.The contrastive objective aligns image features with the quality reasoning text space.Source: §4 · p. 5
  • PCA reduces 768-dimensional image embeddings to 512 dimensions, then bucketed k-means compresses them into score-dispersed representatives.Samples are grouped within score buckets, and each cluster is represented by its mean embedding and mean score.Source: Figure 5 · p. 6, §4 · p. 6
  • For scoring, RALI projects an image into the compressed space, softmax-normalizes cosine similarities to basis vectors, and predicts a weighted sum of their scores.At inference, fixed basis vectors and scores reduce computation to dot products.Source: §4 · pp. 6–7

5 EXPERIMENTS

Experiments evaluate RALI’s score regression, RACT’s cross-domain training, component ablations, and inference efficiency. Results indicate competitive single-domain scoring, strong out-of-domain performance with RACT, and substantially lower inference costs for RALI.

5.1 EXPERIMENTAL SETUP

  • Evaluation spans four IQA dataset groups, with MOS rescaled to [1, 5] and score regression measured by PLCC and SRCC.RALI uses CLIP-ViT-Large-Patch14-336, while RACT uses Qwen-2.5-VL-7B-Instruct.Source: §5.1 · p. 7

5.2 RESULTS OF SCORE REGRESSION

  • SFT methods outperform RL-based counterparts in-domain, while RACT-optimized RL achieves the highest performance across all out-of-domain datasets.The comparison covers co-training on KonIQ, SPAQ, KADID, and PIPAL, with in-domain and out-of-domain test groups.Source: §5.2 · p. 7, Table 3 · p. 8
  • RALI matches Q-Insight competitively on PLCC and SRCC with about 4% of its parameters, and exceeds CLIP-IQA+ by 0.056 PLCC and 0.059 SRCC.Table 2 compares single-domain score regression methods trained on KonIQ, except handcrafted methods.Source: §5.2 · p. 7, Table 2 · p. 8

5.3 ABLATION STUDIES

  • Removing contrastive alignment, PCA, bucketed k-means, or seed augmentation degrades RALI; score definition further improves prediction accuracy.Using standard k-means substantially degrades performance because its cluster-based scores are overly concentrated; removing PCA causes a slight drop.Source: §5.3 · pp. 8–9, Table 4 · p. 9
  • Adding scores to cross-domain SFT benefits in-domain performance but not out-of-domain results, while visual-encoder-only and joint training perform comparably.Cross-dataset score annotations carry subjective annotator biases; visual-encoder-only training converges more slowly.Source: §5.3 · p. 9, Table 5 · p. 9

5.4 EFFICIENCY STUDIES OF RALI

  • At batch size 16 on an NVIDIA A100 (80GB), RALI uses 14.7% of Q-Insight’s memory and 3.4% of its inference time.Figure 6 compares the efficiency of Q-Insight and RALI.Source: §5.4 · p. 9

6 CONCLUSION

The paper attributes reasoning-based IQA models’ generalization to compressing visual information into compact, domain-bridging text. It presents RACT and RALI as complementary approaches, with RACT targeting out-of-distribution performance and RALI offering comparable performance without explicit reasoning.

6 CONCLUSION

  • RACT delivers state-of-the-art out-of-distribution performance on mixed training, while RALI achieves comparable performance with 0.3B parameters and no explicit reasoning.RALI combines contrastive image-text alignment, PCA dimensionality reduction, and bucketed k-means label-text conversion.Source: §6 · p. 9

A THE RELATIONSHIP BETWEEN REASONING AND GENERALIZATION IN Q-INSIGHT

Appendix experiments and discussion examine how reasoning relates to generalization in Q-Insight. Disabling reasoning or using excessively short reasoning is associated with worse performance, especially out of distribution.

A THE RELATIONSHIP BETWEEN REASONING AND GENERALIZATION IN Q-INSIGHT

  • Disabling Q-Insight’s reasoning capability causes a significant performance drop, especially on out-of-distribution testing, despite GRPO modestly improving over SFT.The comparison is reported in Table A.1.Source: A THE RELATIONSHIP BETWEEN REASONING… · p. 15, Table A.1 · p. 15
  • The discussion argues that removing reasoning leaves score-based constraints and forfeits the model’s reasoning advantage; excessively short reasoning also leads to performance drops.The cited discussion of Visual-RFT similarly describes reasoning as important to self-learning and improvement during reinforcement fine-tuning.Source: A THE RELATIONSHIP BETWEEN REASONING… · p. 15

B LIMITATIONS AND BROADER IMPACTS

The paper identifies the CLIP vision encoder and a natural-image focus as boundaries of its current work. It also discusses broader vision-language applications and potential deployment and reward-model uses.

B LIMITATIONS AND BROADER IMPACTS

  • RALI’s performance ceiling is constrained by the representational and reasoning capacity of its CLIP vision encoder, and experiments primarily target natural-image IQA.The authors propose exploring stronger CLIP variants and suggest extension to video and AIGC quality assessment.Source: B LIMITATIONS AND BROADER IMPACTS · p. 15
  • The authors describe the framework as a pathway for converting reasoning-based evaluators into reasoning-free ones, with potential for on-device deployment and reward-model post-training pipelines.They also identify attention, compact textual representations, and cross-domain bias mitigation as potentially useful for broader vision-language research.Source: B LIMITATIONS AND BROADER IMPACTS · p. 15
  • The RACT illustration presents single-domain RL, reasoning-text label alignment, and cross-domain SFT, using one dataset’s scores to stabilize convergence.Cross-domain SFT adapts the visual encoder across domains using aligned image-text pairs.Source: Figure A.1 · p. 16

C MORE DETAILS ABOUT REASONING-ALIGNED CROSS-DOMAIN TRAINING FRAMEWORK.

RACT separates cross-dataset image-to-description training from single-dataset description-to-score training, and the section compares its mixed-training behavior and score-source robustness with Q-Insight.

  • RACT trains image-to-description conversion using annotations from multiple datasets, but trains description-to-score prediction using one dataset’s annotations.The authors report that adding scores from multiple datasets to SFT degrades performance.Source: C MORE DETAILS ABOUT REASONING-ALIGNED… · p. 16
  • The authors attribute multi-dataset SFT degradation to domain-varying annotator-group bias, while RL-derived quality reasoning text is described as objective and cross-domain generalizable.They illustrate that unrelated groups’ preferences are not better fit by combining biases from two groups rather than one.Source: C MORE DETAILS ABOUT REASONING-ALIGNED… · p. 16
  • Mixed training degrades Q-Insight more as datasets are added, while RACT converges and generalizes better, with larger gains as more datasets are included.Table A.2 reports PLCC / SRCC comparisons across training settings and datasets.Source: C MORE DETAILS ABOUT REASONING-ALIGNED… · p. 16, Table A.2 · p. 16
  • RACT is reported to be insensitive to which dataset supplies training scores, with comparisons using numeric scores from KonIQ or SPAQ.Table A.3 covers PLCC / SRCC when RACT is trained with numeric scores from different datasets.Source: C MORE DETAILS ABOUT REASONING-ALIGNED… · pp. 16–17, Table A.3 · p. 17
  • Single-dataset RL prompts pair an image with MOS ground truth, while multi-dataset SFT prompts pair an image with quality reasoning text as ground truth.The task-specific prompts are collected in Table A.4.Source: C MORE DETAILS ABOUT REASONING-ALIGNED… · p. 17, Table A.4 · p. 17

D MORE ABLATION STUDIES

RALI’s hyperparameter study links basis capacity and bucket coverage to score-fitting performance and out-of-domain generalization, leading the authors to select case (8).

  • Too-low PCA dimensions or too few basis vectors limit the post-score-definition performance ceiling because the reduced bases cannot adequately fit the feature space.The tested 256-D / 100 settings and 512-D case (8) reach roughly 0.745 PLCC without score definition.Source: D MORE ABLATION STUDIES · p. 17
  • Too few buckets concentrate basis scores and fail to cover the full score range, while 700-D bases improve in-domain performance but reduce out-of-domain generalization.Excessive dimensionality and many bases also make RALI optimization more difficult.Source: D MORE ABLATION STUDIES · p. 17
  • The authors choose case (8) as RALI’s final solution after weighing these dimension, basis-count, bucket, and optimization trade-offs.Source: D MORE ABLATION STUDIES · p. 17

E RELATION TO MODEL LIGHTWEIGHTING

The appendix contrasts RALI’s efficiency and performance with lightweighted MLLMs, compares reasoning visualizations, and states the extent of LLM use in preparing the paper.

E RELATION TO MODEL LIGHTWEIGHTING

  • RALI achieves up to 95% lightweighting while maintaining performance comparable to MLLMs; Q-Insight’s performance drops significantly after an approximately 50% parameter reduction.The controlled comparison uses a 3B Qwen-VL trained on KonIQ; the table reports PLCC / SRCC on single-domain score regression tasks.Source: E RELATION TO MODEL LIGHTWEIGHTING · pp. 17–18, Table A.6 · p. 18

F VISUALIZATION

G LLM USAGE STATEMENT

  • The authors state that LLM use was limited to minor grammar and phrasing polishes; technical content was created by the authors.Source: G LLM USAGE STATEMENT · p. 18

Related papers

Loading 2510.11369v2…