PaperlayerView exact revision on arXiv

Source-linked AI summary

Specificity-aware reinforcement learning for fine-grained open-world classification

Samuele Angheben, Davide Berasi, Alessandro Conti, Elisa Ricci, Yiming Wang

arXiv:2603.03197v3cs.CV

TL;DR

Fine-grained open-world classification requires correct, specific predictions, yet reasoning LMMs tend to be generic and specificity-promoting methods can harm correctness. SpeciaRL uses verifier-guided dynamic rewards anchored to the best online-rollout predictions, achieving the best specificity–correctness trade-off across out-of-domain fine-grained benchmarks.

  • Problem

    Reasoning LMMs often produce overly generic fine-grained predictions, while increasing specificity can reduce correctness, making their balance a non-trivial challenge.

  • Method

    SpeciaRL fine-tunes reasoning LMMs with a dynamic, sample-wise reward based on the best predictions from online rollouts and graded LLM-verifier feedback within a GRPO-like framework.

  • Results

    SpeciaRL consistently achieves the best trade-off between specificity and correctness across out-of-domain fine-grained and very fine-grained datasets.

  • Takeaways & Limitations

    SpeciaRL promotes specificity within the model’s inherent capabilities while preventing the correctness degradation observed with existing approaches.

Abstract

from arXiv · show

Classifying fine-grained visual concepts under open-world settings, i.e., without a predefined label set, demands models to be both accurate and specific. Recent reasoning Large Multimodal Models (LMMs) exhibit strong visual understanding capability but tend to produce overly generic predictions when performing fine-grained image classification. Our preliminary analysis reveals that models do possess the intrinsic fine-grained domain knowledge. However, promoting more specific predictions (specificity) without compromising correct ones (correctness) remains a non-trivial and understudied challenge. In this work, we investigate how to steer reasoning LMMs toward predictions that are both correct and specific. We propose a novel specificity-aware reinforcement learning framework, SpeciaRL, to fine-tune reasoning LMMs on fine-grained image classification under the open-world setting. SpeciaRL introduces a dynamic, verifier-based reward signal anchored to the best predictions within online rollouts, promoting specificity while respecting the model's capabilities to prevent incorrect predictions. Our out-of-domain experiments show that SpeciaRL delivers the best trade-off between correctness and specificity across extensive fine-grained benchmarks, surpassing existing methods and advancing open-world fine-grained image classification. Code and model are publicly available at https://github.com/s-angheben/SpeciaRL.

1. Introduction

Open-world fine-grained classification requires predictions that are both correct and specific, but reasoning LMMs often produce generic outputs. SpeciaRL addresses this tension with dynamic rewards designed to increase specificity without sacrificing correctness.

  • Open-world classification removes the fixed vocabulary assumption, requiring models to handle emerging categories and novel concepts.
  • Reasoning LMMs perform strongly overall but struggle with fine-grained concepts and tend to make overly generic predictions.
  • Prompting, supervised fine-tuning, and reinforcement fine-tuning can increase specificity but reduce correctness.
  • The model’s generic tendency is not necessarily due to missing domain-specific knowledge; analysis instead examines whether models can produce more specific predictions.
  • SpeciaRL uses a specificity-aware dynamic reward based on the highest specificity level achieved across multiple rollouts, targeting the model’s capabilities.
  • SpeciaRL achieves the best trade-off between specificity and correctness compared with existing methods.

2. Related Work

Research on multimodal models has progressed from aligned visual-text representations to generative and reasoning LMMs, while open-world image-classification evaluation remains comparatively underexplored. SpeciaRL builds on verifier-based reinforcement learning and GRPO to optimize open-ended classification predictions.

  • Generative LMMs connect pretrained visual encoders to language-model input spaces, enabling open-ended visual question answering and visual reasoning.
  • Chain-of-thought prompting and reasoning LMMs use multi-step reasoning to improve performance on complex tasks, while reinforcement learning serves as a post-training strategy.
  • Evaluating LMMs as image classifiers is challenging because their outputs are unconstrained, and this specific evaluation problem has received less attention.
  • Verifier-based evaluation assesses and categorizes relationships between open-ended predictions and ground-truth labels.
  • RLVR replaces subjective human feedback with rule-based or programmatically verifiable rewards, while model-based verifiers extend it beyond structured-solution tasks.
  • SpeciaRL proposes a verifier-based reinforcement-learning framework for open-world image classification that is compatible with on-policy methods such as GRPO.

3. Method

This section formulates open-world fine-grained image classification, evaluates prediction correctness and specificity through LLM-based categorization, and motivates SpeciaRL from models’ generic predictions despite latent fine-grained capability. It then presents the dynamic reward strategy and preliminary evidence supporting its design.

  • 3.1. Problem formulation: Open-world classification maps images to semantic concepts without a predefined or finite class set, allowing labels at multiple hierarchical specificity levels.The semantic space includes concepts expressible through natural-language labels, such as a golden-winged warbler, warbler, or bird.
  • 3.1. Problem formulation: Existing reasoning LMMs often produce correct but generic predictions, while prompting or fine-tuning for specificity can increase wrong outputs.The paper frames the central challenge as improving specificity without compromising correctness.
  • 3.2. Prediction Evaluation: Predictions are categorized by an LLM judge into Wrong, Abstain, Generic, Less Specific, Specific, or More Specific according to their relation to the fine-grained ground truth.The judge uses precise category descriptions and Llama3-72B to assess fine-grained semantic relations.
  • 3.2. Prediction Evaluation: Correctness is the percentage of non-Wrong predictions, while specificity averages category scores that increase from Generic through Specific and More Specific.The assigned scores are s(A)=1, s(G)=2, s(S−)=3, and s(S)=s(S+)=4; harmonic mean combines specificity and correctness.
  • 3.3. On LMMs being overly generic: BoN-64 yields substantially greater specificity and correctness than one-time inference, indicating that the model can produce better predictions within multiple rollouts.The preliminary analysis reports that very fine-grained data produce almost 75% Generic predictions in ordinary inference, whereas the best of 64 rollouts performs better on both measures.
  • 3.4. Specificity-aware Reinforcement Learning: SpeciaRL uses GRPO with a sample-specific verifier reward anchored to the highest specificity achieved across rollouts, rewarding maximal specificity while preserving correctness.Wrong predictions receive zero reward; predictions are rewarded when their category is at least as informative as the adaptive reference level.

4. Experiments

SpeciaRL is evaluated against zero-shot and training-based baselines on out-of-domain fine-grained benchmarks, with additional comparisons, qualitative analyses, and reward ablations. It achieves the strongest reported balance between specificity and correctness while dynamic rewards and moderate rollout sizes support performance.

  • Experimental setup: Training uses 3,000 randomly selected CUB samples, while testing evaluates fine-grained and very fine-grained datasets from different domains.Baselines include retrieval-based CaSED, reasoning LMMs, prompting, supervised fine-tuning, and reinforcement fine-tuning.
  • Main comparison: SpeciaRL achieves the highest harmonic mean across both out-of-domain test groups, with less correctness compromise than competing approaches.On the fine-grained set, it improves both specificity and correctness over the base Qwen2.5VL-7B model.
  • Main comparison: On the benchmark protocol of [9], SpeciaRL reaches state-of-the-art performance on three of four metrics for both test groups.The reported metrics are Text Inclusion, Language Inclusion, Semantic Similarity, and Concept Similarity.
  • Qualitative analysis: Qualitative examples show that both models capture fine visual details, but SpeciaRL more often converts them into fine-grained class predictions.The comparison visualizes reasoning traces, answers, and verifier-assigned prediction categories.
  • Ablation studies: SpeciaRL obtains the best harmonic mean among static-reward variants, while the binary S+&S(1) reward performs worst.The ablation highlights the value of rewarding correct predictions that are less informative than the ground truth.
  • Ablation studies: With N = 5 rollouts, performance resembles the default N = 10 setting; N = 15 lowers both specificity and correctness.N = 5 yields a minor specificity gain and correctness drop, producing equal HM values to N = 10.

5. Conclusion

The paper finds that reasoning LMMs are overly generic in fine-grained open-world classification despite possessing relevant domain knowledge. SpeciaRL uses specificity-aware reinforcement learning to promote the most specific predictions within the model’s capabilities while preserving correctness.

  • Reasoning LMMs are overly generic because they fail to reliably express their most specific available prediction, rather than lacking domain knowledge.
  • SpeciaRL uses a dynamic, sample-wise reward based on the best predictions found during online rollouts.An LLM verifier provides graded feedback within a GRPO-like policy optimization framework.
  • SpeciaRL promotes specificity within the model’s inherent capability while preventing the correctness degradation observed with existing approaches.
  • Across fine-grained and very fine-grained out-of-domain sets, SpeciaRL achieves the best trade-off between specificity and correctness.

Supplementary Material

The supplementary material adds implementation details, complete per-dataset evaluations, and further analyses that complement the main document.

  • The supplementary material provides additional details and analyses complementing the main document.
  • A. Additional implementation details: Section A covers implementation details, including LMM and LLM-verifier prompts and optimization strategies for training efficiency.
  • B. Per-dataset evaluation: Section B reports complete out-of-domain results for individual datasets in fine-grained and very fine-grained sets, with further analyses.

A. Additional implementation details

The supplementary implementation details document the prompts, verifier, training-data construction, and computational optimizations used in the experiments.

  • The experiments use classification, verification, and reasoning-trace prompts for the LMM, LLM-as-a-judge, and supervised fine-tuning baseline.
  • Prompts: The default classification prompt requests reasoning followed by a single label in <think>/<answer> format.The structured output simplifies final-prediction extraction and subsequent verifier checking.
  • Prompts: The “Be specific” baseline adds an explicit specificity requirement, while the format-free protocol follows the prompting strategy of [9].
  • LLM-as-a-judge verifier: The LLM verifier categorizes predictions against ground truth using predefined categories, in-context examples, and guided decoding.The verifier prompt replaces placeholders with the ground truth and JSON-formatted prediction.
  • Supervised fine-tuning baseline: A ground-truth-conditioned prompt generates reasoning traces for each training sample to construct the supervised fine-tuning dataset.
  • Efficiency: vLLM, continuous batching, prefix caching, and persistent verification caching reduce computational costs for LMM inference and LLM-as-a-judge evaluation.The cache stores prediction–ground-truth pairs with their verification categories for reuse across computations and runs.

B.1. Per-dataset evaluation

Across individual out-of-domain datasets, SpeciaRL performs strongly, achieving the best harmonic mean on three of five benchmarks and the second best on the other two.

  • SpeciaRL achieves the best harmonic mean on three of five benchmarks: Flowers102, Food101, and FGVAircraft.
  • SpeciaRL achieves the second-best harmonic mean on OxfordPets and StanfordCars.
  • On Flowers102, OxfordPets, and StanfordCars, SpeciaRL improves both specificity and correctness relative to the base model.
  • The datasets span domains significantly different from the CUB training set, while SpeciaRL remains strong across all evaluation benchmarks.These results support a general classification behavior oriented toward specificity and correctness.

B.2. In-domain evaluation

In-domain evaluation on CUB shows training-based methods achieve very high specificity, while RL-based methods provide the strongest harmonic mean despite remaining below BoN-64 in correctness. These results suggest SpeciaRL learns both more specific and more correct classification behavior.

  • In-domain evaluation: Training-based variants exceed BoN-64 in specificity on the CUB test split.This in-domain evaluation uses models trained on the CUB subset and tested on CUB.
  • In-domain evaluation: Only RL-based methods improve correctness over the base model, although they remain below BoN-64.
  • In-domain evaluation: The evaluation reports specificity, correctness, and harmonic mean across the in-domain training strategies.
  • In-domain evaluation: The two RL-based approaches obtain the best harmonic mean, surpassing BoN-64.The results indicate that models adjust both specificity and correctness and acquire domain-specific information.

B.3. Additional qualitative results

Qualitative results show SpeciaRL more consistently produces specific classifications and uses fine-grained visual evidence, while also revealing wrong predictions and inappropriate scientific names as failure cases.

  • Additional qualitative results: SpeciaRL consistently produces more specific classifications across Flowers102, Food101, OxfordPets, FGVCAircraft, and StanfordCars.Examples are shown two per test dataset, with datasets ordered from Flowers102 through StanfordCars.
  • Additional qualitative results: SpeciaRL reasoning traces more frequently reference fine-grained visual evidence supporting predictions than the Qwen2.5VL-7B base model.The base model sometimes identifies a specific label during reasoning but outputs a generic final label.
  • Failure cases: The qualitative failures include wrong specific predictions and scientific names used for generic concepts such as “Felis Catus” for “Cat.”The verifier categorizes these unusual scientific-name predictions as Generic.
  • Additional prompting results: Additional prompting variants show performance variation, but their overall impact is smaller than the gains from training-based methods.The variants were generated by requesting optimal predictor prompts given the full task context.

B.5. Additional ablation studies

The extended ablations vary training domains, dataset scale, mixed-domain configurations, RL optimizers, and verifier settings to test the robustness of SpeciaRL.

  • Additional ablation studies: The ablations vary training domain, dataset scale, and mixed-domain setups.
  • Additional ablation studies: The study evaluates SpeciaRL across multiple on-policy RL algorithms rather than attributing improvements to one optimizer.
  • Additional ablation studies: Verifier validation includes agreement analyses across models and judge prompts, plus sensitivity tests with injected judge classification errors.

B.5.1. training-data configurations

Additional studies show that SpeciaRL benefits from more and more representative training data, transfers across on-policy optimizers, and remains robust to moderate verifier noise, while verifier agreement varies by model.

  • Training-data configurations: In-domain models generally perform best on their corresponding test domains when trained on Flowers102, Food101, or OxfordPets.The study compares models trained independently on these three fine-grained datasets.
  • Training-data configurations: Increasing the CUB training-set size raises both specificity and correctness in in-domain evaluation.The main comparisons use a 3000-sample CUB training subset with otherwise identical epochs and hyperparameters.
  • Training-data configurations: An in-domain balanced mixture of samples from all evaluation domains outperforms the out-of-distribution CUB-trained model.The mixture includes 500 samples from each of six evaluation domains, including CUB.
  • RL algorithms configuration: SpeciaRL improves specificity, correctness, and harmonic mean across GRPO, Dr.GRPO, and DAPO, with harmonic-mean gains from +0.015 to +0.058.The results indicate compatibility with general online RL frameworks rather than dependence on one policy-optimization algorithm.
  • Verifier validation: Qwen3-30B shows almost perfect agreement with Llama3-72B, whereas Llama3-7B shows moderate agreement.Llama3-72B is also reported as insensitive to variations in the judge prompts.
  • Verifier validation: SpeciaRL remains rather robust for verifier error rates ρ_e ≤10%, while higher noise levels begin degrading the training signal.
Loading 2603.03197v3…