NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: Professional Image Quality Assessment (Track 1)

1. Introduction

• Conventional IQA compresses complex visual quality into scalar MOS, limiting its practical value for distinguishing uniformly high-quality images or explaining why one is better.This lack of reasoning also limits the feedback such methods can provide for vision models.Source: §1 · p. 1
• MLLMs may overcome IQA’s interpretability bottleneck by combining visual perception and language generation to express grounded comparisons of subtle textures, noise, and artifacts.Although MLLMs show promise in quality-score regression, their potential value extends beyond producing scores to emulating expert cognition.Source: §1 · pp. 1–2
• The challenge provides expert-annotated high-quality photo pairs and targets reliable superior-image selection plus grounded explanations of quality differences.Its broader aim is to bridge expert cognition and automated assessment, providing interpretable guidance for photography and downstream restoration tasks.Source: §1 · p. 2

2.1. Training Data

• The 100-pair training set uses expert-selected A/B comparisons with textual rationales and subject crops to assess photographic attributes; participants may add data, but not manually annotated data or closed-source VLM annotations.It contains 75 portrait pairs and 25 landscape or scenery pairs, with diverse subjects, environments, and lighting conditions.Source: §2.1 · pp. 2–3

2.2. Validation and Test Data

• Validation (102 pairs) and test (101 pairs) use subjects and landscape scenes excluded from earlier splits, providing a zero-overlap assessment of generalization.Validation includes two unseen subjects; test adds two more unseen subjects, and its scenery pairs come from scenes unused in training.Source: §2.2 · p. 4

2.3.1. Comparative Accuracy

• Evaluation measures both comparative accuracy and explanation quality; accuracy is the share of pairs whose predicted A/B choice matches the ground truth under case-insensitive string matching.The answer must be enclosed in <answer> and </answer> tags.Source: §2.3 · p. 4, §2.3.1 · p. 4

2.3.2. Reasoning Quality Evaluation

• Reasoning quality combines conditional text metrics with an independent semantic judge: BLEU-4 and ROUGE-L are averaged only for correct selections, while the judge scores alignment to expert reasoning on a 0–1 scale.The judge evaluates whether the rationale reflects photographic advantages such as sharpness, texture, realism, and noise.Source: §2.3.2 · p. 4

2.3.3. Ranking Metrics

• Phase 2 ranking combines accuracy with conditioned reasoning quality, Phase 3 adds the semantic-judge score, and the final ranking averages the two phase scores.The weighting keeps accuracy central while rewarding reasoning, with an additional semantic score in Phase 3.Source: §2.3.3 · pp. 4–5

2.4.3. Phase 3: Final Evaluation

• The challenge proceeds from model design using training data, to online validation feedback, and then to a single-submission final test evaluated after organizer verification and LLM judging.Phase 2 returns accuracy and an integrated score through the public leaderboard; Phase 3 withholds annotations and limits the submission window.Source: §2.4.1 · p. 5, §2.4.2 · p. 5, §2.4.3 · p. 5

2.5. Awards

• Track awards comprise one US$1000 first-class prize, two US$500 second-class prizes, and three US$200 third-class prizes.The final award ranking is determined from Phase 2 and Phase 3 scores.Source: §2.5 · p. 5, Table 1 · p. 6

2.6. Important Dates

• The announced schedule runs from Phase 1 data release on 2026.01.28 through final rank announcement on 2026.03.20.Phase 2 data was released on 2026.02.08, Phase 3 began on 2026.03.10, and its submission deadline was 2026.03.12.Source: §2.6 · p. 6

3. Challenge Results

• The challenge attracted 192 registrations; 85 participants submitted 2,530 entries in Phase 2, and 57 teams advanced to Phase 3 with one submission each.Phase 3 enforced a strict single-attempt rule.Source: §3 · p. 6

3.1. Quantitative Comparison

• IH-VQA ranked first in Final Award Score, while VCIP Pi Group led Phase 3 through a notably high LLM Score; top-team gaps stayed narrow, and balanced capabilities mattered across criteria.The results span all three phases.Source: §3.1 · p. 6

3.2. Qualitative Comparison

• Models generally made binary selections proficiently, but often failed to align their rationales with nuanced photographic dimensions identified by experts.Assessing reasoning across image contents and shooting conditions remains open because optimal criteria vary with visual context and artifacts.Source: §3.2 · p. 6

NTIRE 2026 Team

• Figure 4 compares generated rationales and selections with ground-truth annotations and selections on the portrait portion of the curated test set.Source: Figure 4 · p. 7
• Figure 5 compares generated rationales and selections with ground-truth annotations and selections on the scenery portion of the curated test set.Source: Figure 5 · p. 7

5 Computer Vision Lab, University of W¨urzburg, Germany

• The affiliation listings include Northwestern Polytechnical University, Xi’an University of Architecture and Technology, Chongqing University, and the University of Science and Technology of China.Source: §5 · p. 8

4.1. Submitted Methods

• Participating teams commonly used GRPO and voting ensembles, with detailed team methods deferred to supplementary material because of space limits.Source: §4.1 · p. 8

5. Conclusion

• The challenge established a benchmark of high-quality image pairs with expert-annotated reasoning, moving evaluation beyond simple numerical regression.It was designed to address limitations of conventional scalar-based IQA.Source: §5 · p. 8
• Top-performing methods demonstrated MLLMs’ viability for robust comparative quality selection and grounded interpretative reasoning.The paper presents these results as evidence that MLLMs can bridge automated assessment and human cognition.Source: §5 · p. 8
• The authors hope the datasets and multi-tiered metrics will inspire a shift from scoring toward industrially valuable evaluation systems.The intended applications include next-generation photography and vision.Source: §5 · p. 8

A. Team & Methods

• The section provides a brief overview of participating teams and their proposed methods for the track.Source: A. Team & Methods · p. 12

A.1. IH-VQA - Methodology

• The framework pairs an Answer Model for robust binary preference prediction with a Thinking Model for rationale generation.The branches target accurate decision making and interpretability, respectively.Source: §A.1 · p. 12
• The Answer Model focuses on accurate binary decisions about which image is preferred.Source: §A.1 · p. 12
• The Thinking Model produces structured textual explanations for predicted preferences to improve interpretability.Source: §A.1 · p. 12

A.1.1. Data Synthesis and Pre-processing

• The Answer Model receives four separate global and cropped views and is trained as separate person and scene models to capture relative quality differences and reduce content heterogeneity.Thinking Model instruction data is progressively enriched with expert-style templates, traditional CV features, learned IQA features, and answer-aware prompts.Source: §A.1.1 · p. 12

A.1.2. Network Design

• The Answer Model uses shared-backbone features from four decomposed views for left-right preference prediction, with architecture ensembles and separate person and scene branches to improve generalization and model domain-specific cues.Source: §A.1.2 · p. 12
• The Thinking Model generates comparison rationales from image pairs and structured prompts, progressively adding domain-specific templates, CV statistics, learned IQA metrics, and answer-aware refinement.The metrics include Q-Align, LIQE, and SAMA; these prompt enhancements substantially improve explanation quality and consistency.Source: §A.1.2 · p. 12

• The Answer Model’s binary classifiers train for 10 epochs with AdamW, a 1 × 10−4 learning rate and weight decay, batch size 32, 256 × 256 resizing, 224 × 224 random crops, and 5-fold training.Source: §A.1.3 · p. 12
• Fold-wise voting and a multi-backbone ensemble are used to produce robust final Answer Model predictions.Source: §A.1.3 · p. 12
• The Thinking Model fine-tunes Qwen3-VL-8B-Instruct with LoRA-based supervised fine-tuning for 6 epochs at 4 × 10−5 learning rate, using rank r = 16 and DeepSpeed ZeRO-1 on 8 GPUs.Source: §A.1.3 · p. 12

A.2. VCIP Pi Group - Methodology

• VCIP Pi Group uses a Qwen3-VL core agent to coordinate nine dimension-specific tools in a closed-loop comparison pipeline.The Planner selects tool-region calls, the Executor gathers tool outputs and updates memory, and the Summarizer produces the final multidimensional comparison. Training uses SFT then GRPO for the tools and GRPO for the core agent, with rewards for format, answer accuracy, tool format, and tool recall.Source: §A.2 · p. 12, §A.2.2 · p. 13, §A.2.3 · p. 13

A.3. I2 Group - Methodology

• The I2 Group progressively trains Qwen3-VL-8B-Instruct and majority-votes three expert checkpoints for pairwise assessment.It splits merged detailed images into single-view A/B crops, builds original and instruction-expanded SFT sets plus original and swapped-order GRPO sets, and trains through SFT, three GRPO stages, an SFT refresh, and final GRPO; checkpoints from Stages 1, 2, and 5 form the ensemble.Source: §A.3 · p. 13, §A.3.1 · p. 13, §A.3.2 · p. 13

A.3.4. Validation Performance

• The I2 Group’s final three-expert ensemble achieved 0.8627 accuracy and a final score of 0.649 on Phase 2 validation.Evaluated on 102 image pairs, the ensemble was the best-performing stage-wise model, and progressive training consistently improved validation accuracy.Source: §A.3.4 · p. 15, Table 4 · p. 15

A.4. fugui - Methodology

• The fugui team separates binary answer prediction from rationale generation using two task-specific branches on a shared backbone.Its data augmentation includes ground-truth image effects, noise, contrast changes, rotations, A/B swapping, and localized crops; the answer branch omits reasoning text, while the thinking branch explicitly elicits it.Source: §A.4 · p. 15, §A.4.1 · p. 15, §A.4.2 · p. 15

A.5. LZ - Methodology

• LZ combines SFT and GRPO on official data, then ensembles predictions from seven GRPO checkpoints.It begins with two SFT checkpoints and runs five GRPO tasks; all seven saved GRPO checkpoints independently process validation and test data before their outputs are aggregated and post-processed.Source: §A.5.1 · p. 15, §A.5.2 · p. 15, §A.5.3 · pp. 15–16

A.6. ongaku - Methodology

• Ongaku masks thinking tokens from the loss so training emphasizes final assessment accuracy rather than stylistic variation in reasoning.Its Qwen3-VL workflow uses geometric augmentation, image swapping, and random crops; validation and test crops are generated consistently with the training distribution.Source: §A.6 · p. 16, §A.6.1 · p. 16, §A.6.2 · p. 16

A.7. test123djak - Methodology

• test123djak independently trains answer and thinking models, separating final comparative labels from rationale supervision.The team splits concatenated images into high-resolution frames, augments through metadata swapping and horizontal flipping, and keeps the vision model and multimodal aligner unfrozen during training.Source: §A.7 · p. 16, §A.7.1 · p. 16

A.8. dr0strange - Methodology

• dr0strange predicts image preference with LightGBM over six NR-IQA metrics, then uses an MLLM to explain the predicted choice.The metrics combine statistical, regression-based, and deep perceptual cues; explanations are conditioned on the images and decision and address factors such as sharpness, noise, texture fidelity, and contrast.Source: §A.8 · p. 17, §A.8.1 · p. 17, §A.8.2 · p. 17